Proxy – Access Control Methods

When you think initially about access control to a standard proxy one of the most obvious options is tradtional user name and password. Indeed access control by user authentication is one of the most popular methods if only because it’s generally one of the simplest to implement. Not only does it use readily available information for authentication it will also fit neatly in with most corporate networks which generally run on a Windows or Linux platforms. All common OS’s support user authentication as standard and normally using a variety of protocols.

Access control based on the username and group is a commonly deployed feature of proxies. It requires users to authenticate themselves to the proxy server before allowing the request to pass. This way, the proxy can associ- ate a user identity with the request and apply different restrictions based on the user. The proxy will also log the username in its access log, allowing logs to be analyzed for user-specific statistics, such as how much bandwidth was consumed by each user. This can be vital in the world of high traffic multimedia applications and a few users using your Remote access server as a handy BBC VPN service can bring a network to it’s knees.

Authentication There are several methods of authentication. With HTTP, We/9 servers support the Basic authentication, and sometimes also the Digest authentication (see HTTP Authentication on page 54). With HTTPS—— or rather, with any SSL-enhanced protocol—certificate-based authentication is also possible. However, current proxy servers and clients do not yet support HTTPS communication to proxies and are therefore unable to perform certificate-based authentication.

This shortcoming will surely be resolved soon. Groups Most proxy servers provide a feature for grouping a set of users under a single group name. This allows easy administration of large numbers of users by allowing logical groups such as admin, engineering, marketing, sales, and so on. It will also be useful in multinational organisations where individuals may need to authenticate in different countries and using global user accounts and groups. So if a UK based salesman was travelling in continental Europe he could use his UK account to access a French proxy and use local resources.

ACCESS CONTROL BY CLIENT HOST ADDRESS An almost always used access control feature is limiting requests based on the source host address. This restriction may be applied by the IP address of the incoming request, or the name of the requesting host. IP address restrictions can often be specified with wildcards as entire network sub- nets, such as 112.113.123 . * Similarly, wildcards can be used to specify entire domains: * .

Access control based on the requesting host address should always be performed to limit the source of requests to the intended user base.

So what’s a Digital Certificate?

We’ve probably all seen those simple diagrams where an electronic signature authenticates the key pair used to create the signature. For electronic commerce, authenticating a key pair might not be adequate. For business transactions, each key pair needs to be closely bound to the consumer that owns the key pair. An electronic certificate is a credential that contrasts an integral pair to the entity that owns the key pair. Digital certificates are issued by certification authorities,
therefore we trust the binding prescribed with the certificate.

A Digital signature is fine for verifying e-mail, but stronger verification methods are needed to associate an individual, like the demonstration in our earlier post where we used it to allow access to an app for watching the BBC News abroad. to the binary bits on the network that are purporting to “belong” to Tom Smith. For electronic commerce to work, the association has to be of a power that is legally binding. When Tom Smith has an electronic certificate to advertise to the planet at large, he is in possession of something which might take more trust than the “seal” that was made by his own digital signature.

You might trust his digital signature, but what if a few other believed authority had trust in Tom Smith?
Wouldn’t you then trust Tom Smith a little more? A digital certificate is given by an organization that has a reputation to defend. This organization, known as the certificate authority (CA), may be Tom’s employers, an independent organization, or the government. The CA will take measures to set some truths about Torn Smith before issuing a certificate because of him.

The certificate will normally hold Tom’s name, his public key number, the serial number of the certificate itself, and validity dates (issue and expiry). It’ll also bear the name of the issuing CA. The whole certificate is digitally signed by the CA’s own private key.

Lastly we’ve achieved a mechanism which may be used to allow individuals who’ve no previous relationship to set each other’s identity and participate in the legal transactions of electronic commerce. It’s certainly more efficient and secure than using something like geo-location which simply determines your identity based on your location. So for example, a web site might determine nationality by using your network address – e.g a British IP address needed to access the BBC online.

Certificates, if delivered correctly, inspire trust among Internet traders. It’s not, however, as easy as it might sound.
Certificates expire, are missing, are issued to the wrong person, or have to be revoked because the detail held on the certificate is wrong–maybe the people key number was threatened–and this leads to a large Certificate Control effort or even a campaign.

The X.509 v3 certificate format is a standard used for public important certificates and is broadly used by Internet security protocols (like SHTTP). Based on X.509 v3, digital certificates are being used increasingly as electronic credentials for identification, non- repudiation, and even authorization, when making payments and conducting other business transactions on the Internet or corporate Intranets.

Just as within our credit card system of today, where millions of credit card numbers issued by any bank in the world are electronically confirmed, so it will be the use of digital certificates will demand a clearing house network for certificate confirmation of a comparable scale.

Single proxies or proxy arrays?

If you’re working in a small business or network then this issue will probably never arise. However with the growth of the internet and web enabled devices and clients it’s an issue that will almost certainly effect most network administrators. Do we just keep adding an extra proxy to expand capability and bandwidth or should you install an array.

Nevertheless the solution can be dependent on a variety of external factors. for example in the event the corporation is concentrated in a single location, just one level of proxies is a better solution. This reduces the latency as there’s only a single additional hop added by proxies, as opposed to two or more with tree structured proxy hierarchies.

Although the general rule would be to have one proxy server for every 5000 (possible, not simultaneous) users, it doesn’t automatically mean that a company with 9000 users should have 3 departmental proxies, that are then chained to some most important proxy.

Instead, the 3 proxies might be installed in parallel, using Cache Array Routing Protocol (CARP) or another hashbased proxy selection mechanism. Larger corporations with in-house programming skills may have resource to create custom solutions too which work better to a specific environment which perhaps incorporates remote VPN access to the network too. For example many larger environments have different levels of security in place and have various zones which need to be isolated, generic ‘serve all’ proxies can be a significant security issue in these environments.

This approach can also combine multiple physical proxy caches into a single logical one. Ln general, such clustering of all proxies is recommended as it increases the effective cache size and eliminates redundancy between individual proxy caches. Three proxies, each with a 4 gigabyte cache, would give an efficient 12 gigabytes of cache when put up in parallel,as opposed to only about 4GB if used individually.

Generally, some quantity of parallelization of proxies into arrays is obviously desired. Nevertheless, the network layout might dictate that depart psychological proxies be utilized. That is, it is not feasible to have all of the trafc originating from the entire company go through one array of proxies. It can cause the entire array to become a 1/ O bottleneck, even when the individual proxies of the variety have been in individual subnets. The load created by the users can be so high that the subnets leading to the proxies may choke. To alleviate this, some departmental proxies need to be deployed closer to the end customers, in order that a number of the traffic created by the users will not reach the main proxy array.

Failover? Since proxies are a centralized point of traffic it’s vitally important that there is a system in place for failover. If a proxy goes down, users will instantly
lose their access to the internet. What’s more it may be that many important applications rely on permanent internet access to keep running. They might need access to central database systems or perhaps need frequent updates or security patches. In any ways, internet access is often much more crucial than simply the admin office being able to use Amazon, surf UK TV abroad or check the TV schedules online.

Failover might be achieved in many various ways. There are (relatively expensive) hardware solutions which transparently change to a hot standby system in the event the primary system goes down.

Nevertheless, proxy autoconfiguration and CARP provide more cost effective failover support. During the time of this writing, there are a couple areas in customer failover sup port which might be improved. Users have a Propensity to detect a intermediate proxy server going down by seeing fairly long delays, and possibly error messages. A proper proxy back up system should be virtually seamless and provide similar levels of speed and bandwidth than the primary system.

Code Signing – How it Works

How do you think that users and computers can trust all this random software which appears on large public networks?  I am of course referring to the internet and the requirement most of us have to download and run software or apps on a routine basis.  How can we trust that this is legitimate software and not some shell of a program just designed to infect our PC or steal our data?  After all even if we avoid most software, everyone needs to install driver updates and security patches.

The solution generally involves something called code signing which allows companies to assure the quality and content of any file released over the internet.    The software is signed by a certificate and as long as you trust the certificate and it’s issuer then you should  be happy to install the associated software.    Code signing is used by most major distributors in order to ensure the quality of released software online.

Code Signing – the Basics
Coed signing simply adds a small digital signature to a program, an executable file, an active X control, DLL (dynamic link library) or even a simple script or java applet. The crucial fact is that this signature seeks to protect the user of this software in two ways:

Digital signature identified the publisher, ensuring you know exactly who wrote the program before you install it.

Digital signature allows you to determine whether the code you are looking to install is the same as that was released. It also helps to identify what if any changes have been made subsequently.

Obviously if the application is aware of code signing this makes it even simpler to use and more secure. These programs can be configured to interact with signed/unsigned software depending on particular circumstances. One simple example of this are the security zones defined in Internet Explorer. They can be configured to control how each application interacts depending on what zone they are in. There can be different rules for ‘signed’ and ‘unsigned’ applications for instance with obviously more rights assigned to the ‘signed’ applications.

In secure environments you can assume that any ‘unsigned’ application is potentially dangerous and apply restrictions accordingly. Most web browsers have the ability to determine the difference between these applications and assign security rights depending on the status. It should be noted that these will be applied through any sort of connection or access, even a connection from a live VPN to watch the BBC!

This is not restricted to applications that operate through a browser, you can assign and control activity of signed and unsigned applications in other areas too.  Take for instance device drivers, it is arguably even more important that these are validated before being installed.  You can define specific GPO settings in a windows environment to control the operation and the installation of a device driver based on this criteria.

As well as installation it can control how Windows interacts with these drivers too,  although generally for most networks you should not allow installation of an unsigned driver.  This is not always possible though, sometimes application or specialised hardware will need device drivers where the company hasn’t been able to sign the code satisfactorily.   In these instance you should consider carefully before installing and consider the source too. For example if you have downloaded from a reputable site using a high anonymous proxies to  protect your identity then that might be safer than a random download from an insecure site, there is still a risk though.

What Is VPN?

The remote server would access the request, then authenticate through something like a username and password. The tunnel would be established and used to transfer data between the client and server.

If you want to emulate a point to point link, the data must be wrapped with a header – this is normally called encapsulation. This header should provide essential routing information which enables the data to traverse the public network and reach it\’s intended endpoint. In order to keep the link private on this open network all the data would normally be encrypted. Without this route information the data would never reach it\’s intended destination. The encryption ensures that all data is kept confidential. Packets that are intercepted on the shared or public network are indecipherable without the encryption keys. The link in which the private data is encapsulated and encrypted is known as a VPN connection.

One of the most important uses of remote access VPN connections is that it allows workers to connect back to their office or home by using the shared infrastructure of a public network such as the internet. At the users point, the VPN establishes an invisible connection between the client and the organisation’s servers. There is normally no need to specify and aspects of the shared network as long as it is capable of transporting traffic, the VPN tunnel controls all other aspects.   This does mean it’s very difficult to block these VPN connections as the BBC is discovering.

These connections are also known as router to router connections which are established between two fixed points. They are normally setup between distinct offices or based again using the public network of the internet. The link will operate in a similar way to a dedicated wide area network link, however at a fraction of the costs of a dedicated line. Many companies use these increasingly in order to establish fixed connections without the expense of WAN connections. It should be noted that these VPN connections operate over the data link layer of the OSI model.

One of the problems many network administrators find is that users on networks can set up their own VPN connections.  These can be very difficult to detect and allow a direct tunnels into a corporate network especially as they are often used for trivial issues such as obtaining an IP address for Netflix.  Needless to say having users stream encrypted videos streams to their desktops is not good for network performance or security.

Remember a site to site connection will establish a link between two distinct private networks. The VPN server will ensure that a reliable route is always available between the two VPN endpoints. One of the routers will take the role of the VPN client, by requesting the connection. The second server will authenticate and then reciprocate the request in order for the tunnel to be authenticated at each end. In these site to site connections, the packets which are sent across the routers will typically not be created on the routers but clients connected to these respective devices.


Using Reverse Proxies in your Environment

Many IT administrators use proxies extensively in their networks, however the concept or reverse proxying is slightly less common.  So what is a reverse proxy? Well, it refers to the setup where a proxy server like this is run in such a way that it appears to clients just like a normal web server.

Specifically, the client will connect directly to the proxy considering it to be the final destination i.e. the web server itself, they will not be aware that the requests could be relayed further to another server.   It is possible that this will even be an additional proxy server.   These ‘reverse proxy servers’ are also often referred to as gateways although this term can have different meanings too.  To avoid confusion we’ll avoid that description in this article.

In reality the word ‘reverse’ refers to the backward role of the proxy server. In a standard proxy, the server will act as a proxy for the client initially.  Any request by the proxy is made on behalf of the received client request.  This is not the case in the ‘reverse’ scenario because because it acts as a proxy for the web server and not the client.  This distinction can look quite confusing, as in effect the proxy will forward and receive requests to both the client and server however the distinction is important.  You can read RFC 3040 for further information on this branch of internet replication and caching.

A standard proxy is pretty much dedicated to the client’s needs,  all configured clients will forward all their requests for web pages to the proxy server.   In a standard network architecture they will normally sit fairly close to the clients in order to reduce latency and network traffic.   These proxies are also normally run by the organisations themselves although some ISPs will offer the service to larger clients.

In the situation of a reverse proxy, it is representing one or a small number of origin servers.  You cannot normally access random servers through a reverse proxy because it has to be configured to specifically access certain web servers.  Often these servers will need to be highly available and the caching aspect is important,  a large organisation like Netflix would probably have specific IP addresses (read this) pointing at reverse proxies.  The list of servers that are accessible should always be available from the reverse proxy server itself.   A reverse proxy will normally be used by ‘all clients’ to specifically access certain web resources, indeed access may be completely blocked by any other route.

Obviously in this scenario it is usual for the reverse proxy to be both controlled and administered by the owner of the origin web server.  This is because these servers are used for two primary purposes to replicate content across a wide geographic area and two replicate content for load balancing.  In some scenarios it’s also used to add an extra layer of security and authentication to accessing a secure web server too.


Do I Need a Residential or Datacenter IP Address

For many people, there is a very strong requirement to mask their true identity and location online.  IT might be for privacy reasons, perhaps to keep safe or you simply don’t want anyone to log everything you do online.  There are other reasons, using multiple accounts on websites, IP bans for whatever reason or simple region locking – no you can’t watch Hulu on your holidays in Europe. The solution usually now revolves around hiding your IP address using a VPN or proxy as a minimum.

Yet the choice doesn’t end there, proxies are pretty much useless now for privacy and security.  They’re easily detected when you logon and to be honest of very little use anymore.   VPN services are much better, yet even here it’s becoming more complicated to access media sites for example.   The problem is that it’s not the technology that is now the issue but the originating IP address. These are actually classified into two distinct groups – residential and commercial which can both be detected by most websites.


A residential IP address is one that appears to come from a domestic account assigned from an ISP. It’s by far the most discrete and secure address to use if you want to keep completely private. Unfortunately these IP addresses are difficult to obtain in any numbers and also tend to be very expensive. Bottom line is the majority of people for whatever reason who are hiding their true IP address do it by using commercial addresses and not residential ones.

Most security systems can easily detect whether you are using a commercial or residential vpn service provider, how they use that information is a little more unsure. So at the bottom of the pile for security and privacy are the datacentre proxy servers which add no encryption layer and are tagged with commercial IP addresses.

Do I really need a residential VPN IP Address? Well that depends on what you are trying to achieve, for running multiple accounts on things like craigslist and Itunes – residential is best. If you want to try and access the US version of Netflix like this, then you’ll definitely need a residential address. Netflix last year filtered out all commercial addresses which means that very few of the VPNs work anymore, and you can’t watch Netflix at work either.

If you just want to mask your real IP address then a commercial VPN is normally enough. The security is fine and no-one can detect your true location, although they can determine you’re not a home user if they check. People who need to switch IPs for multiple accounts and using dedicated tools will probably be best advised to investigate the residential IP options.

On-Demand Caching for Proxies

It is arguably the most important function of a web proxy at least as far as performance is concerned and that’s on-demand caching.  That is documents or web pages which are cached upon request by a client or application.    It’s important to remember that a document can only be cached if it has actually been requested by a user.  Without a request, it will not be cached and indeed the proxy server will not even be aware of it’s existence.

This is a different method than using a replication model which is typically used to distribute data and updates.   This is more often used on larger, busier networks where data can be replicated onto specific servers, this method is also known as mirroring and also useful for sharing over the internet.     One of the most common examples for mirroring is when a large software package is being distributed instead of a single server being responsible, multiple duplicates are replicated onto different servers.

One of the best ways to facilitate performance increases is to use a method called round-robin DNS.  This involves mapping a single host name to multiple physical servers.  These servers must be assigned separate IP and physical addresses and their addresses distributed evenly among the clients requesting the software.    When using the DNs method, the clients will be unaware of the existence of multiple servers because they will appear as a single logical server.

Most of the caching solutions used by proxies are centred around removing the load on a specific server.  However when a proxy caches resources locally without mirroring or replication then it’s still the single server which is responsible.    The physical loads doesn’t decrease however it does reduce the number of network requests that the server has to implement.   This also reduces the number of name requests that the server makes which can also introduce some levels of latency.

Having caching enabled can reduce the speed of the server responses significantly.  However this does depend largely on the sort of requests that are made, imagine a proxy used specifically to obtain a Czech IP Address and directly download a specific resource.  Caching that resource locally would improve the speed significantly as long as the content didn’t change much, however this would be different for sites which stream audio or video and contained large amounts of multimedia content.

Further Reading



Proxy Configuration – SSL Tunneling

The SSL Tunneling Protocol allows any proxy server which supports it the ability to act as a tunnel for SSL enhanced protocols.  This feature is essential to support normal web traffic and increasingly SSL is being used to secure normal web requests which would previously have been sent in clear text.    The client makes the initial HTTP request to the proxy and asks for an SSL tunnel.  If we look at the protocol level the actual handshake to establish the SSL tunneling connection is fairly straight-forward.

The connection is simple and in fact looks like virtually any other HTTP request, the only difference is that we use a new ‘Connect’ method.   The format is also slightly different as the parameter is not a full url but rather the destination host address and the post number in the format      The port number is always required with these connection requests, as the default number is generic and not always correct.

When the client has received a successful response then the connection will pass all data in both directions to the destination server.  For the proxy server much of it’s role in authentication and establishing the connection is over, and it’s role is then limited to simply forwarding data for the connection.   The final significant role for the proxy server is to close the connection which it will do when it receives a close request from either the client or the server.

Other situations where the connection will be closed mainly refer to error status codes.  For example an error generated in response to authentication would be a typical situation where authentication has failed.  Most proxies will require some sort of authentication especially the high quality US proxies such as this.  The methods might change however from a simple username password supplied via a challenge and response to pass through authentication from a system like the Active Directory or LDAP.

It’s interesting to note that the mechanism used to handle SSL tunnelling is not actually specific to this protocol.  It is in fact a generic technique which can be used to tunnel any protocol including SSL.   There is no actual reliance on any SSL support from the proxy, which can be confusing when you see people look for SSL enabled proxies online.    It is not required on a properly configured proxy server as the data is simply transported there is no need for the actual protocol to be understood after the initial connection request.

There are issues with some protocols transferring through proxies, certain specialised protocols need more support than is offered with the standard tunneling mechanism.  For example for many years LDAP (Lightweight Directory Access Protocol) was not able to work across most common proxies.  Some implementations support LDAP by using SOCKS while there is some difficulty with LDAP queries being cached and subsequently causing performance issues.    Most protocols however work perfectly with this ‘hands off’ tunneling mechanism which you can see perfectly illustrated if you try and stream video through proxies like this which used to circumvent BBC iPlayer blocked abroad.


Protecting Your Network with IDS

There is little excuse for not installing an IDS (Intrusion Detection System) on your Network, even the usual culprit of budget doesn’t apply.  In fact one of the leading IDS systems called Snort is actually available completely free of charge and is sufficient for all but the most very complex network infrastructures.    It is virtually impossible to effectively monitor and control your network, particularly if it’s connected to the internet, without some sort of IDS in place.

There are certain questions about the day to day operation of your network that you should be able to answer.  Questions like the following will help you determine if you really have control over your network and it’s hardware =

  • Can you tag and determine how much traffic on  your network is associated with malware or unauthorised software.
  • Are you able to determine which of your clients do not have the latest client build?
  • Can you determine which websites are most popularly requested.  Are these requests from legitimate users or as a result of malware activity.
  • Can you determine which users are the top web surfers (and is it justified).
  • How much mail are your SMTP server’s processing?

It is surprising how many network professionals simply wouldn’t have a clue about obtaining this information from their network however, it’s impossible to ensure that the network is efficient without it.  For example a few high intensive web users can create much more traffic than the majority of ordinary business users.   Imagine two or three users in a small department who used a working BBC VPN to stream TV to their computer 8 hours a day.   The traffic that would generate would be huge and could easily swamp an important network segment.

All security professionals should ensure that they have the tools and reporting capacity to answer simple questions like this about network usage.   Knowing the answers to these questions, will help control and adapt your network to meet it’s users needs.   Of course a simple IDS won’t provide the complete solution but it will help keep control in your network.  Malware can sit and operate for many weeks in a network which is not monitored properly.  This will heavily impact performance and can enable it to spread to other devices and eventually other networks.  In network environments where performance is important, then being aware of the sorts of situations can make a huge difference.

John Williams

Network Professional and Broadcaster on author of BBC News Streaming.