It is arguably the most important function of a web proxy at least as far as performance is concerned and that’s on-demand caching. That is documents or web pages which are cached upon request by a client or application. It’s important to remember that a document can only be cached if it has actually been requested by a user. Without a request, it will not be cached and indeed the proxy server will not even be aware of it’s existence.
This is a different method than using a replication model which is typically used to distribute data and updates. This is more often used on larger, busier networks where data can be replicated onto specific servers, this method is also known as mirroring and also useful for sharing over the internet. One of the most common examples for mirroring is when a large software package is being distributed instead of a single server being responsible, multiple duplicates are replicated onto different servers.
One of the best ways to facilitate performance increases is to use a method called round-robin DNS. This involves mapping a single host name to multiple physical servers. These servers must be assigned separate IP and physical addresses and their addresses distributed evenly among the clients requesting the software. When using the DNs method, the clients will be unaware of the existence of multiple servers because they will appear as a single logical server.
Most of the caching solutions used by proxies are centred around removing the load on a specific server. However when a proxy caches resources locally without mirroring or replication then it’s still the single server which is responsible. The physical loads doesn’t decrease however it does reduce the number of network requests that the server has to implement. This also reduces the number of name requests that the server makes which can also introduce some levels of latency.
Having caching enabled can reduce the speed of the server responses significantly. However this does depend largely on the sort of requests that are made, imagine a proxy used specifically to obtain a Czech IP Address and directly download a specific resource. Caching that resource locally would improve the speed significantly as long as the content didn’t change much, however this would be different for sites which stream audio or video and contained large amounts of multimedia content.
The SSL Tunneling Protocol allows any proxy server which supports it the ability to act as a tunnel for SSL enhanced protocols. This feature is essential to support normal web traffic and increasingly SSL is being used to secure normal web requests which would previously have been sent in clear text. The client makes the initial HTTP request to the proxy and asks for an SSL tunnel. If we look at the protocol level the actual handshake to establish the SSL tunneling connection is fairly straight-forward.
The connection is simple and in fact looks like virtually any other HTTP request, the only difference is that we use a new ‘Connect’ method. The format is also slightly different as the parameter is not a full url but rather the destination host address and the post number in the format 192.168.1.1:8080. The port number is always required with these connection requests, as the default number is generic and not always correct.
When the client has received a successful response then the connection will pass all data in both directions to the destination server. For the proxy server much of it’s role in authentication and establishing the connection is over, and it’s role is then limited to simply forwarding data for the connection. The final significant role for the proxy server is to close the connection which it will do when it receives a close request from either the client or the server.
Other situations where the connection will be closed mainly refer to error status codes. For example an error generated in response to authentication would be a typical situation where authentication has failed. Most proxies will require some sort of authentication especially the high quality US proxies such as this. The methods might change however from a simple username password supplied via a challenge and response to pass through authentication from a system like the Active Directory or LDAP.
It’s interesting to note that the mechanism used to handle SSL tunnelling is not actually specific to this protocol. It is in fact a generic technique which can be used to tunnel any protocol including SSL. There is no actual reliance on any SSL support from the proxy, which can be confusing when you see people look for SSL enabled proxies online. It is not required on a properly configured proxy server as the data is simply transported there is no need for the actual protocol to be understood after the initial connection request.
There are issues with some protocols transferring through proxies, certain specialised protocols need more support than is offered with the standard tunneling mechanism. For example for many years LDAP (Lightweight Directory Access Protocol) was not able to work across most common proxies. Some implementations support LDAP by using SOCKS while there is some difficulty with LDAP queries being cached and subsequently causing performance issues. Most protocols however work perfectly with this ‘hands off’ tunneling mechanism which you can see perfectly illustrated if you try and stream video through proxies like this which used to circumvent BBC iPlayer blocked abroad.
There is little excuse for not installing an IDS (Intrusion Detection System) on your Network, even the usual culprit of budget doesn’t apply. In fact one of the leading IDS systems called Snort is actually available completely free of charge and is sufficient for all but the most very complex network infrastructures. It is virtually impossible to effectively monitor and control your network, particularly if it’s connected to the internet, without some sort of IDS in place.
There are certain questions about the day to day operation of your network that you should be able to answer. Questions like the following will help you determine if you really have control over your network and it’s hardware =
- Can you tag and determine how much traffic on your network is associated with malware or unauthorised software.
- Are you able to determine which of your clients do not have the latest client build?
- Can you determine which websites are most popularly requested. Are these requests from legitimate users or as a result of malware activity.
- Can you determine which users are the top web surfers (and is it justified).
- How much mail are your SMTP server’s processing?
It is surprising how many network professionals simply wouldn’t have a clue about obtaining this information from their network however, it’s impossible to ensure that the network is efficient without it. For example a few high intensive web users can create much more traffic than the majority of ordinary business users. Imagine two or three users in a small department who used a working BBC VPN to stream TV to their computer 8 hours a day. The traffic that would generate would be huge and could easily swamp an important network segment.
All security professionals should ensure that they have the tools and reporting capacity to answer simple questions like this about network usage. Knowing the answers to these questions, will help control and adapt your network to meet it’s users needs. Of course a simple IDS won’t provide the complete solution but it will help keep control in your network. Malware can sit and operate for many weeks in a network which is not monitored properly. This will heavily impact performance and can enable it to spread to other devices and eventually other networks. In network environments where performance is important, then being aware of the sorts of situations can make a huge difference.
Network Professional and Broadcaster on author of BBC News Streaming.
For many people, travel is becoming much easier and as a species our geographical horizons are perhaps wider than ever. Inexpensive air travel and soft borders like the European Union means that instead of just looking to work in another city or town, another country is just as viable. The internet of course enables this somewhat, many corporations have installed infrastructure to allow remote or home working which means many people can work from wherever they wish. Instead of sitting in cubicles in vast expensive office space, the reality is that people can work together just as easily using high speed internet connections from home.
Unfortunately there are some issues from this digital utopia, of which most are self inflicted. Instead of being a vast unfettered global communications medium, the internet in some senses has begun to shrink somewhat. Not so much in size but rather an increasing number of restrictions, filters and blocks being applied to web servers across the planet. For instance the company I work for has two main bases one in the UK and the other in Poland, which means there is quite a bit of travel between the two countries. Not surprisingly employees who are working away from home for some time, use the internet to keep in touch with their homelife, yet this can be frustrating.
A common issue is the fact that many websites are not really accessible globally, they are locked to specific regions. Take for example the main Polish TV channel – TVN, it has a fantastic website and a media player by which you can watch all their shows. However a Polish citizen who tries to watch the local News from Warsaw from a hotel in the UK will find themselves blocked, the content is only available to those physically located in Poland. It’s no one off either, this behaviour is shared by pretty much every large media company on the web who block access depending on your location.
There is a solution and for our employees it’s actually quite simple, all they need to do is fire up their VPN client and remotely connect back to their home server in Poland. The instant they do this, their connection looks like it’s based in Poland and all the Polish TV channels will work perfectly. There’s a post about something similar here – using a Polish proxy to watch TVN and some other channels although this one is through a commercial service designed to hide your location. It’s a practice that is becoming increasingly necessary, the more we travel the more we find our online access is determined by our physical location.
The use of proxies and more recently VPNs allows you to break out of these artificial intranets which companies are creating by blocking access from other countries. The idea is that if you have the ability to switch to various VPNs across the world you can effectively take back control and access whatever website you need. Your physical location becomes unimportant again, by taking control of your virtual location you have an huge advantage over other internet users by choosing the location you wish to appear from. There are even some other options now take a look at this UK DNS proxy which does something fairly similar and can be used to watch the BBC and Netflix from outside the UK.
Author of – Does BBC Iplayer Work in Ireland
In these times when security is becoming ever more important the SSL Tunneling Protocol is extremely important, it allows a web proxy server to act as a tunnel for SSL enhanced protocols. The protocol is used when any connected client makes a HTTP request to the proxy server and asks for a SSL tunnel to be initiated. On the HTTP protocol level, the handshake required to initiate the SSL tunneling connection is simple. There is little difference to an ordinary HTTP request except that a new ‘Connect’ method is used and the parameter passed is not a full URL but instead a destination port number and hostname separated by a colon.
The port number is always required with ‘CONNECT’ requests because the tunneling method is generic and there is no protocol specified, hence default port numbers cannot be used reliably. The general syntax for the request is as below ;
CONNECT <host>:<port> HTTP/1.0
HTTP Request Headers
The successful response would be a connection established message, followed by another empty line. After the successful response the connection will then pass all the data transparently to the destination server and pass through any replies from the server. In practice what is happening is the proxy is validating the initial request, establishes the connection and then takes a step back. After this initial stage the proxy merely forwards data back and forth between the client and the server. If either side closes the connection then the proxy will cause both connections to be closed and no mor tunneling will take place until a new connection is established between the server and client.
The proxy does have the ability to respond to error messages within the SSL tunnel. If this error is generated in the initial stages then the connection will not be established, if it is already connected then the proxy will close the connection after the error response has been sent. However it is important to remember especially where security is important that this SSL tunneling protocol is not specific to SSL and therefore has no in depth security. The tunnelling mechanism used in this instance is a generic one and can in fact be used for any protocol. This means that there is no requirement either for the proxy to support SSL either as the server is merely establishing a connection and then forwarding data without any processing.
BBC Iplayer Ireland – Here’s How you Can Watch