• Download source - 25.56 KB
• Download demo - 11.42 KB  
• HTTP_Spy-src.zip HTTP_Spy-demo.zip

Introduction   

A simple way to find the configuration of a Web Server is by using any one of the HTTP request methods HEAD and GET. When an HTTP request is made for a resource on the Web Server, in response, the Server along with other information sends its own configuration.

The HTTP Response from Web Server contains meta-information in the form of HTTP headers and the message body. The application parses the response for the “Server” header field, which contains the server configuration. The application can also find if a server permits HTTP HEAD requests and the difference in response for different User Agents.

The name and functionality of this tool was inspired by the “HTTP spy” utility provided in 3d Traceroute (http://www.d3tr.de/httpserverspy.html).

Background

RFC 2616 defines Hypertext Transfer Protocol (HTTP) as an application-level protocol for distributed, collaborative, hypermedia information systems. HTTP requests are used for retrieving files from a Web Server. HTTP is a text based protocol consisting of requests and related responses.

HTTP Methods

The most common HTTP request methods are GET and POST. The GET method is used to get a resource from the HTTP Server while POST can send information to the web server. HEAD is identical to GET, except that the former doesn't expect the server to send message body in response. The HEAD method can be used for obtaining meta-information about the entity implied by the request without transferring the entity-body itself. This method is often used for testing hypertext links for validity, accessibility, and recent modification. 

“HTTP Spy” can send HTTP HEAD and GET request methods to the Web Server. Some web server hosts (e.g. www.msn.com) do not permit HTTP HEAD requests. However, almost all web servers support the GET request, which is the most popular/default method used by HTTP clients (especially by Browsers). The GET and HEAD methods don't have the significance of taking any action other than retrieval, so are considered "safe".

HTTP Client Functionality

The underlying transport protocol that does the communication on behalf of HTTP is TCP, which is a connection-oriented protocol. A connection would require the server host name and port number, on which the Web Server application would be waiting for servicing requests from clients. For HTTP the most commonly used port no is 80. However, any free port can be used, and in this case, usually needs to be specified explicitly in the HTTP URI. For example if www.site.com accepts HTTP connections on port 8080 then the HTTP request URI should specify this port number. The HTTP port is specified just after a colon “:”, which follows the hostname (i.e. www.site.com:8080). If the HTTP port is not specified explicitly then by default the HTTP client will use port 80. This would result in TCP connection error, in case the web server is not listening on port 80. However, HTTP Spy doesn't make this a rule, i.e. the HTTP port can be mentioned in the HTTP port edit box instead of explicitly specifying it in the URI.

The application can change its identify (to different browsers or bots), depending on the User Agent selection. This can be used for checking the difference (if any) in the response from a server for the different User Agents.

HTTP Proxy Support

The new version supports connecting to HTTP servers through the HTTP proxy. HTTP proxy is usually used to restrict web access in an enterprise LAN. The main purpose is to allow a single public IP address to be shared among all LAN users for accessing the internet.

To use this feature, the HTTP proxy IP address and its port number should be specified, along with the HTTP server to be queried. The application makes TCP connection to the proxy server and sends the HTTP request to it. The HTTP proxy, on behalf of the user forwards the request to the Web Server. The resulting response from Web Server is then conveyed back to the user by the proxy server.

Implementation

Class HttpSocket has methods that implement HTTP methods. It creates a TCP connection to the HTTP server or proxy server as required by the application. Before that, it parses the HTTP URI, to separate the host, HTTP port information (if specified) and the requested resource on the web server. This is later used to construct the HTTP headers for the request.

A separate child thread performs the HTTP processing for each request. After the completion of the request, the child thread exits.

Conclusion

The application assumes that the meta-information, send by the web server contains its configuration (in the HTTP, nevertheless, there can be exceptions (e.g. www.yahoo.com).