The Concise Guide to Proxies
We often mention that the benefits derived from some application delivery controllers are due to the nature of being a full proxy. And in the same breath we might mention reverse, half, and forward proxies, which makes the technology sound more like a description of the positions on a sports team than an application delivery solution. So what does these terms really mean? Here's the lowdown on the different kinds of proxies in one concise guide. PROXIES Proxies (often called intermediaries in the SOA world) are hardware or software solutions that sit between the client and the server and do something to requests and sometimes responses. The most often heard use of the term proxy is in conjunction with anonymizing Web surfing. That's because proxies sit between your browser and your desired destination and proxy the connection; that is you talk to the proxy while the proxy talks to the web server and neither you nor the web server know about each other. Proxies are not all the same. Some are half proxies, some are full proxies; some are forward and some are reverse. Yes, that came excruciatingly close to sounding like a Dr. Seuss book. (Go ahead, you know you want to. You may even remember this from .. .well, when it was first circulated.) FORWARD PROXIES Forward proxies are probably the most well known of all proxies, primarily because most folks have dealt with them either directly or indirectly. Forward proxies are those proxies that sit between two networks, usually a private internal network and the public Internet. Forward proxies have also traditionally been employed by large service providers as a bridge between their isolated network of subscribers and the public Internet, such as CompuServe and AOL in days gone by. These are often referred to as "mega-proxies" because they managed such high volumes of traffic. Forward proxies are generally HTTP (Web) proxies that provide a number of services but primarily focus on web content filtering and caching services. These forward proxies often include authentication and authorization as a part of their product to provide more control over access to public content. If you've ever gotten a web page that says "Your request has been denied by blah blah blah. If you think this is an error please contact the help desk/your administrator" then you've probably used a forward proxy. REVERSE PROXIES A reverse proxy is less well known, generally because we don't use the term anymore to describe products used as such. Load balancers (application delivery controllers) and caches are good examples of reverse proxies. Reverse proxies sit in front of web and application servers and process requests for applications and content coming in from the public Internet to the internal, private network. This is the primary reason for the appellation "reverse" proxy - to differentiate it from a proxy that handles outbound requests. Reverse proxies are also generally focused on HTTP but in recent years have expanded to include a number of other protocols commonly used on the web such as streaming audio (RTSP), file transfers (FTP), and generally any application protocol capable of being delivered via UDP or TCP. HALF PROXIES Half-proxy is a description of the way in which a proxy, reverse or forward, handles connections. There are two uses of the term half-proxy: one describing a deployment configuration that affects the way connections are handled and one that describes simply the difference between a first and subsequent connections. The deployment focused definition of half-proxy is associated with a direct server return (DSR) configuration. Requests are proxied by the device, but the responses do not return through the device, but rather are sent directly to the client. For some types of data - particularly streaming protocols - this configuration results in improved performance. This configuration is known as a half-proxy because only half the connection (incoming) is proxied while the other half, the response, is not. The second use of the term "half-proxy" describes a solution in which the proxy performs what is known as delayed binding in order to provide additional functionality. This allows the proxy to examine the request before determining where to send it. Once the proxy determines where to route the request, the connection between the client and the server are "stitched" together. This is referred to as a half-proxy because the initial TCP handshaking and first requests are proxied by the solution, but subsequently forwarded without interception. Half proxies can look at incoming requests in order to determine where the connection should be sent and can even use techniques to perform layer 7 inspection, but they are rarely capable of examining the responses. Almost all half-proxies fall into the category of reverse proxies. FULL PROXIES Full proxy is also a description of the way in which a proxy, reverse or forward, handles connections. A full proxy maintains two separate connections - one between itself and the client and one between itself and the destination server. A full proxy completely understands the protocols, and is itself an endpoint and an originator for the protocols. Full proxies are named because they completely proxy connections - incoming and outgoing. Because the full proxy is an actual protocol endpoint, it must fully implement the protocols as both a client and a server (a packet-based design does not). This also means the full proxy can have its own TCP connection behavior, such as buffering, retransmits, and TCP options. With a full proxy, each connection is unique; each can have its own TCP connection behavior. This means that a client connecting to the full proxy device would likely have different connection behavior than the full proxy might use for communicating with servers. Full proxies can look at incoming requests and outbound responses and can manipulate both if the solution allows it. Many reverse and forward proxies use a full proxy model today. There is no guarantee that a given solution is a full proxy, so you should always ask your solution provider if it is important to you that the solution is a full proxy.8 things you can do with a proxy
After having recently discussed all the different kinds of proxies that exist, it occurred to me that it might be nice to provide some examples of what you can do with proxies besides the obvious web filtering scenario. This is by no means an exhaustive list, but is provided to show some of the more common (and cool, I think) uses of proxies. What's really awesome is that while some of these uses are available with only one type of proxy (reverse or forward), a full proxy can provide all these uses, and more, in a single, unified application delivery platform. 1. DATA SCRUBBING Data scrubbing is the process of removing sensitive information like credit card and social security numbers from web application responses. This is particularly useful in preventing data leaks, especially if you're subject to regulations like SOX, HIPPA, and PCI DSS where the penalties for divulging personally identifiable information can be harsh fines - or worse. Data scrubbing is is an implementation of a reverse proxy. 2. URL REWRITING Rewriting URLs is something everyone has likely had to do at one time or another if they've developed a web application. URL rewriting is used to refer web requests to new resources instead of sending out a redirect response in cases where resources have moved, renamed, or migrated to a new version. URL rewriting is an implementation of a reverse proxy. 3. LAYER 7 SWITCHING Layer 7 switching provides an organization with the ability to maximize their IP address space as well as architect a more efficient, better performing application architecture. Layer 7 switching routes specific web requests to different servers based on information in the application layer, like HTTP headers or application data. Layer 7 switching is an implementation of a reverse proxy. 4. CONTENT FILTERING The most common use of proxies is content filtering. Generally, content filtering allows or rejects requests for content based on organizational policies regarding content type, the existence of specific keywords, or based on the site itself. Content filtering is an implementation of a forward proxy. 5. REDIRECTION Redirection is the process of, well, redirecting a browser to a new resource. This could be a new instance of a requested resource or as part of application logic such as redirecting a failed login to the proper page. Redirection is generally implemented by a reverse proxy, but can also be implemented by a forward proxy as a means of redirecting rejected requests to an explanation page. 6. LOAD BALANCING Load balancing is one of the most common uses of a reverse proxy. Load balancing distributes requests for resources across a number of servers in order to provide scalability and availability services. Load balancing is an implementation of a reverse proxy. 7. APPLICATION FIREWALL An application firewall provides a number of functions including some in this list (data scrubbing and redirection). An application firewall sits in front of web applications and inspects requests for malicious content and attempts to circumvent security. An application firewall is an implementation of a reverse proxy. 8. PROTOCOL SECURITY Protocol security is the ability of a proxy to enforce protocol specifications on requests and responses in order to provide additional security at all layers of the OSI stack. Protocol security provides an additional layer of security atop traditional security mechanisms that focus on data. Protocol security is an implementation of a reverse proxy.
He Who Defends Everything Defends nothing… Right?
There has been much made in Information Technology about the military quote: “He Who Defends Everything Defends Nothing” – Originally uttered by Frederick The Great of Prussia. He has some other great quotes, check them out when you have a moment. The thing is that he was absolutely correct in a military or political context. You cannot defend every inch of ground or even the extent of a very long front with a limited supply of troops. You also cannot refuse to negotiate on all points in the political arena. The nature of modern representative government is such that the important things must be defended and the less important offered up in trade for getting other things you want or need. In both situations, demanding that everything be saved results in nothing being saved. Militarily because you will be defeated piecemeal with your troops spread out, and politically because your opponent has no reason to negotiate with you if you are not willing to give on any issue at all. But in high tech, things are a little more complex. That phrase is most often uttered to refer to defense against hacking attempts, and on the surface seems to fit well. But with examination, it does not suit the high-tech scenario at all. While defense in depth is important in datacenter defense, just in case someone penetrates your outer defenses. But we all know that there are one or two key choke-points that allow you to stop intruders who do not have inside help – your Internet connections. If those are adequately protected, the chances of your network being infiltrated, your website taken down, or any of a million other ugly outcomes are much smaller. The problem, in the 21st century, is the definition of “adequate”. Recent attacks have taken down firewalls previously assumed to be “adequate”, and the last several years have seen a couple of spectacular DNS vulnerabilities focusing on a primary function that had seriously seen little attention from attackers or security folks. In short, the entire face you present to the world is susceptible to attack. And at the application layer, attacks can slip through your outer defenses pretty easily. That’s why the future network defensive point for the datacenter will be a full proxy at the Strategic Point of Control where your network connects to the Internet. Keeping attacks from dropping your network requires a high-speed connection in front of all available resources. The Wikileaks attacks took out a few more than “adequate” firewalls, while the DNS vulnerabilities attacked DNS through its own protocol. A device in the strategic point of control between the Internet and your valuable resources needs to be able handle high-volume attacks and be resilient enough to respond to new threats be they at the protocol or application layers. It needs to be intelligent enough to compare user/device against known access allowances and quarantine the user appropriately if things appear fishy. It also needs to be adaptable enough to adapt to new attacks before they overwhelm the network. Zero day attacks by definition almost never have canned fixes available, so waiting for your provider to plug the hole is a delay that you might not be able to afford. That requires the ability for you to work in fixes and an environment that encourages the sharing of fixes – like DevCentral or a similar site. So that you can quickly solve the problem either by identifying the problem and creating a fix, or by downloading someone else’s fix and installing it. While an “official” solution might follow, and eventually the app will get patched, you are protected in the interim. You can defend everything by placing the correct tool at the correct location. You can manage who has access to what, from which devices, when, and how they authenticate. All while protecting against DOS attacks that cripple some infrastructures. That’s the direction IT needs to head. We spend far too many resources and far too much brainpower on defending rather than enabling. Time to get off the merry-go-round, or at least slow it down enough that you can return your focus to enabling the business and worry less about security. Don’t expect security concerns will ever go away though, because we can – and by the nature of the threats must – defend everything.Why Single-Stack Infrastructure Sucks
You may have heard the term “full-proxy architecture” or “dual stacks” thrown around in the context of infrastructure; here’s why that distinction is important. When the terms “acceleration” and “optimization” in relation to application delivery are used it often evokes images of compression, caching, and similar technologies. Sometimes it even brings up a discussion on protocol optimization, which is really where things get interesting. You see, caching and compression techniques are mostly about the content – the data – being transferred. Whether it’s making it smaller (and thus faster) or delivering it from somewhere closer to the user (which also makes it faster) the focus of these acceleration techniques is really the content. But many of the most beneficial optimizations happen below the application data layer, at the transport level and below. It’s all about the stack, baby. A good one is fast, a bad one, well, isn’t. But it isn’t just about an optimized stack. Face it, there are like a gazillion different tricks, tips, and cheats for optimizing the stack on every operating system but all of them are peculiar to a specific operating environment. Which is great if you’re an end-user trying to trick out your desktop to download that ginormous file even faster. When it’s not so great is when you’re a web server or a piece of application delivery infrastructure. ONE SIZE DOES not FIT ALL So here’s the thing – when you tweak out a single-stack piece of infrastructure for a specific environment you’re necessarily ignoring every other environment. You have to pick and choose what set of optimizations you’re going to use, and you’re stuck with it. If eighty percent of your user-base is accessing an application over “link A” then the other twenty percent are probably going to experience poor performance – and you’ll be lucky if they don’t experience time- outs or resets as well. This problem (which has been solved by full-proxy, dynamic dual-stack infrastructure for a long time) has reared its ugly head yet again recently with the excitement over virtual network appliances (VNA). You know, a virtual image of your infrastructure components, deployed in the same flexible, rapid manner as your applications. The problem with this is that just slapping a network component into a virtual image results in a less than optimal integration. The component leverages the networking stack of the hypervisor necessarily, which means it is optimized to communicate over a LAN. A low latency, high-throughput, high capacity network connection without a lot of congestion. You know, the kinds of things that make WAN-delivered applications slow, sluggish, and unresponsive. For the same reasons that a web/application server – regardless of form-factor – can’t be optimized for both LAN and WAN at the same time neither can a VNA. It has a single-stack because that’s what’s underlying the entire system and what’s being interfaced with. It cannot simultaneously address pain points with WAN connected communications and LAN connected communications. So not only are you incapable with a single-stack infrastructure of optimizing and accelerating on a per-connection basis, when you deploy an infrastructure component in virtualized form (or any other form that results in a single network stack architecture) you are now incapable of optimizing and accelerating on a per network connection basis. It’s LAN or WAN, baby. Those are your choices. TRANSLATORS and TRAFFIC COPS An intermediary is defined as a “mediator: a negotiator who acts as a link between parties”. The analogy of a “translator” is often used to describe the technical functionality of an intermediary, and it’s a good one as long as one remembers that a translator actually does some work – they translate one language to another. They terminate the conversation with one person and initiate and manage conversations with another simultaneously. They are “dual” stacked, if you will, and necessarily must be in order to perform the process of translation. This is in stark contrast to previous analogies where load balancers and other application delivery focused infrastructure were analogized as “traffic cops.” Traffic cops, when directing traffic, do not interact or otherwise interrupt the flow of traffic very often. They are not endpoints, they are not involved in the conversation except to point out where and when cars are allowed to go. They do not interact with the traffic in the way that a translator does. In fact they use nearly universal hand signals to direct traffic (think transport protocol layer and below) because they are primarily concerned with speed and performance. Their job is to get that car (packet) moving in the right direction and get it out of the way. They don’t care where its going or what its going to do there; traffic cops only care about making sure the car (packet) is on its way. Translators, intermediaries, care about what is being said and they are adept at ensuring that the conversation is managed properly. Speed and performance are important, but making sure the conversation is accurate and translated correctly is as important to the translator as doing so quickly. Traffic cops are single-stacks; translators are dual-stacks. DIALECTS and DIFFERENCES When you have exactly the same connection type on both sides of the conversation, a traffic cop is okay. But this is almost never the case, because even when two clients access an application over the generic “WAN”, there are still variances in speed, latency, and client capabilities. Sure, they’re both speaking Chinese, but they’re both speaking different dialects of Chinese that each have their own nuances and idioms and especial pronunciation that requires just a bit different handling by the infrastructure. Optimizing and accelerating those connections requires careful attention to each individual conversation, and may further require tweaks and tuning on-demand for that specific conversation over and above the generic WAN-focused tweaks and tuning performed to enhance WAN communication. A dual-stack infrastructure component is an intermediary. It can perform the function of a traffic-cop if that’s all you need but it is almost certainly the case that you need more, because users and partners and integrated applications are accessing your applications from a variety of client-types and a broad set of network connections. Dual-stack infrastructure separates, completely, the client communication from the server-communication, and enables the application and enforcement of policies that enhance security, performance, and availability by adapting in real-time to the conditions that exist peculiar to the client and the application. Single-stack infrastructure simply cannot adapt to the volatile environment of today’s modern deployment architectures, e.g. cloud computing , highly virtualized, multi-site, and highly distributed. Single-stack infrastructure – whether network or server – are unable to properly optimize that single network stack in a way that can simultaneously serve up applications over WAN and LAN, and do so for both mobile and desktop clients such that both are happy with the performance. Consider the queues on your web server – that’s where data collects on a per-connection basis, waiting to be transferred to the client. There are only so many queues that can be in use at any given time – it’s part of the capacity equation. The ability of clients to “pull” that data out of queues is directly related to the speed and capacity of their network connection and on the configuration and resources available of their client. If they pull it too slowly, that queue is tied up and resources assigned to it can’t be used by other waiting users. Slow moving queues necessarily decrease the concurrent user and connection capacity of a server (virtual or iron) and the result necessitates more hardware and servers as a means to increase capacity. A single-stack infrastructure really can’t address this common problem well. A dual-stack infrastructure can, by leveraging its buffering capacity to quickly empty those queues and re-use the resources for other connections and users. In the meantime, its doling out the data to the client as quickly or slowly as the client can consume it, with negligible impact on the infrastructure’s resource availability. Dual-stack infrastructure can be tweaked and tuned and adapts at execution time. It’s agile in its ability to integrate and collaborate with the rest of the infrastructure as well as its ability to apply the right policies at the right time based on conditions that are present right now as opposed to when it was first deployed. It can be a strategic point of control because it intercepts, inspects, and can act based on context to secure and accelerate applications.Full (proxy) Security
With the increasing number of "data leaks" involving large numbers of affected consumers there is an increased focus on products that prevent such leaks from occurring in the first place. Many of these products have grown out of the IDS (Intrusion Detection System) market and others have been built from the ground up. Some, like F5's BIG-IP Application Security Manager (ASM), have grown out of the WAF (Web Application Firewall) product set. So what's the difference between them? One of the biggest differentiators in these product sets is the way in which they are deployed, which is necessitated by their architecture. Products coming out of the IDS space tend to be passive monitors that attempt to recreate streams of data and, upon discovering a potential leak, try to stop the flow of traffic (generally referred to as 'quarantining' the traffic). Products built atop a full proxy architecture, like ASM and F5 BIG-IP Local Traffic Manager (LTM), can be active monitors because its architecture is such that they are part of the flow of traffic and therefore less error-prone than their passively deployed cousins. Products that are full proxies provide inherently better security because they actively terminate the flow of data, essentially creating an "air gap" security model inside the product. Traffic coming from the client can be examined before it is sent on its way to the application tier, ensuring that malicious traffic never passes the proxy barrier. Traffic returning from the server can be fully examined before it is deemed acceptable to pass back to the client, thereby ensuring that sensitive data such as credit-card or social-security numbers are never passed across the proxy barrier. Products built on a passive model don't have the luxury of being able to stop traffic in its tracks because they aren't part of the flow of traffic. They must rely on other mechanisms, such as using a TCP reset, in order to prevent data leaks or stop malicious code from entering the data center. Unfortunately, TCP reset mechanisms can result in a race-condition - that is, will the TCP RST reach the offending client before it receives the real response or will it arrive after, at which point the process has essentially failed. Data leak protection is only one of the many security features that can be offered by full-proxy based solutions. The ability to sit in between the flow of data and inspect the data before allowing it to continue - in either direction - offers the opportunity to apply security in many ways to the data including protocol sanitization, resource obfuscation, and signature-based scanning. It's tempting to give in to the alluring cry of easier configuration and deployment offered by passive monitoring based security systems. A passive deployment is easier, no doubt, but the risks of deploying such a system for the purposes of preventing data leaks are also higher. Consider carefully whether you're willing to accept the risks associated with products based on passive monitoring techniques or if you'd prefer to spend a bit more time up front configuring and deploying a full-proxy based solution in exchange for lowering your risk of being compromised. Imbibing: Water Technorati tags: MacVittie, F5, security, full proxy, data leaks
