Neokernel API

com.neokernel.httpd Namespace

Namespace Hierarchy

Classes

Class Description
ConnectionKMethod This class is a KMethod with all the code for dealing with an incoming HTTP connection. The WebServer creates one of these ConnectionKMethods for each client that connects to the server.
Notes on Keep-Alive connections:
There are 2 fields in the AgentRequest that pertain to keep-alive connections: KEEP_ALIVE and KEEP_ALIVE_FIELD.

The KEEP_ALIVE_FIELD property is set by the ConnectionKMethod, if this property is NOT set, a Keep-Alive connection cannot be used.
The KEEP_ALIVE property is set by the agent that handles the request, if this property is set, the ConnectionKMethod will keep the connection alive.
The agent (for instance, a HTTPFileServerAgent) that handles the request is responsible for the following:
It must return a content-length header in the response, this usually means the agent must return its own HTTP response.
If a keep alive is allowed, and the client wants a keep-alive, the agent must return the Connection: Keep-Alive field as well as the contents of the KEEP_ALIVE_FIELD property.
If a keep alive is allowed, and the client wants a keep-alive, the agent must set the KEEP_ALIVE property in the AgentRequest.
If a keep alive is allowed, and the client wants a keep-alive, the agent must NOT close the Socket or the Socket streams.
ConnectionKMethod.BackInputStream Provides functionality to read and unread from a Stream.
HTMLFormWriter HTMLFormWriter is a tool for writing simple HTML-based forms to a StringBuilder or AgentRequest output stream. This class makes it easier to create generic forms without dealing with lots of HTML.
HTMLStringTools This class has a collection of static methods for HTML/HTTP related string manipulation.
HTTPAgentRequest This file is an extension of the AgentRequest class with specialty fields and methods for handling HTTP requests. It prints the HTTP headers to the response stream if the agent responding to the request doesn't print correct HTTP/1.1 response headers before writing other content to the output stream. HTTPAgentRequest also has methods for using cookies and for managing sessions with a SessionManager. HTTPAgentRequest implements the Props interface, and the following props are usually set by the server:
path
The requested path, for instance /index.html.
request_remote_addr
The address of the machine making the request.
request
The text of the request, for example: GET /index.html HTTP/1.1\nAccept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, application/x-shockwave-flash, application/vnd.ms-excel, application/vnd.ms-powerpoint, application/msword, */*\nAccept-Language: en-us\nAccept-Encoding: gzip, deflate\nUser-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322; .NET CLR 1.0.3705; InfoPath.1)\nHost: localhost\nConnection: Keep-Alive\n
request_server_name
The server name or IP, for instance 172.20.0.108.
request_type
The type of request, for example HTTP.
request_id
The ID assigned to the request by the server, for instance HTTP:client1.
host
The server name requested by the client (useful on machines hosting multiple websites), for instance www.microsoft.com.
request_server_port
The server port that the request arrived on, usually port 80 (or 441 for HTTPS requests).
HTTPAuthenticationAgent This agent is used by the WebServer to handle authorization of password protected resources.

The authentication file defines the realms and the usernames and passwords which are used to authenticate clients to the paths associated with each realm. The authentication file is delimited by newlines and must end with a newline. Each realm is declared with a PATH and the REALM name separated by spaces, followed by usernames and passwords on new lines, separated by a space. PATHs must begin with a slash and end with a slash.

The HOSTNAME field at the beginning of the file indicates to the HTTPAuthentication agent what hostname the realms should be mapped to. In Virtual Host (multihoming) configurations, use multiple authentication files with different HOSTNAME values for each file. The default hostname is default.

Here is an example authentication file:
HOSTNAME
PATH0 REALM0
username0 password0
username1 password1
username2 password2
username3 password3
PATH1 REALM2
username0 password0
username1 password1

The following example shows an example of an authentication file with two realms, 'My realm' and 'Main Realm'. If any access is made to files or directories in /private/, this agent will ask for the username jonlin with the password mainpassword, except for accesses to /private/mine/, which is under the /private/ path but is its own realm and requires its own usernames and passwords. If default is used as the hostname then the authentication map will apply to all virtual hosts on this server.

default
/private/mine/ My realm
john johnpassword
santos halper
/private/ Main Realm
john mainpassword
This agent supports the following Props settings:
service_name
The name this agent uses to register with Service Manager. The WebServer's authentication_agent prop must be set to match this agent's service_name, or requests for urls requiring authentication will not be handed off to this agent. The default value is http_authentication.agent.
authentication_files
The file(s) used for authenticating requests. The default value is ./http.authentication. Separate multiple files with commas.
HTTPCheckKMethod This KMethod checks if a web server is available by sending it a request for a URL. After the KMethod has been executed, the connection, request, and request latency are recorded in public fields. This method requires a host, port and request as arguments to the constructor. An optional timeout field can be set if needed.
HTTPDecoderStream This extension of Stream allows only a fixed number of bytes to be read from the stream before behaving as if the Stream is empty. The bytes that are read from this stream are automatically decoded using HTTP parameter decoding.
HTTPFileServerAgent

This agent is used by the WebServer and SecureWebServer to handle requests for files from the file system.

If the file is an ASP.NET file, it will be executed using the ASP.NET environment on the server and the output will be sent to the requestor. If the ASP.NET code references other assemblies or .dlls, include copies of these .dlls in the same folder as your ASP.NET file.

The HTTPFileServerAgent supports the following Props:

service_name
The name this agent uses to register with Service Manager. The default value is HTTPFileServerAgent
html_directory
The path to the directory where HTML files are served from. Defaults to ./
default_index
The default files (separated by commas) to serve when no filename is specified in the HTTP request. This prop is set to index.html by default, which means requests for /public_html/my_stuff/ will be served the file /public_html/my_stuff/index.html.
allow_directory_lists
If this is set to yes, this agent will display links to all files in a directory when a web request specifies a directory instead of a file. The default value is no
dont_index_list
This is the list of files that shouldn't appear in directory index listings, can use '*' to denote a wildcard. This property is not used if the allow_directory_lists property is set to false. By default, this property isn't set.
alpha_sort_index
If set to true, directory lists will be sorted in alphanumerical order. This property is set to true by default.
aspnet_config_file
The path to an XML file containing ASP.NET configuration and debugging information. Defaults to web.config.
preload_aspx_pages
A list describing the ASP.NET pages that should be pre-loaded when the agent starts. This is not required but it makes pages load faster the first time they are requested by the user. To pre-load ASP.NET pages the agent must know the file location and it's URL relative to the server root. The list of files is in the format:
[WebPath1],[FilePath1];[WebPath2],[FilePath2] (or [WebPath1],[FilePath1],[VirtualDirectory1] if the page is in a virtual directory),
and so on, where the first item in each pair is the path to the file relative to the root of the web server (e.g. /aspapp/myfile.aspx) and the second is the path to the file on the hard disk (e.g. d:\htmldir\aspapp\myfile.aspx). This property is not set by default.
aspx_assemblies
An optional comma separated list of assemblies referenced by ASP.Net pages. Assemblies appearing in this list will be copied to the \bin subdirectory of the folder where the ASP.Net page is located. The files will be re-copied and re-loaded if a newer version becomes available. Use the full path. Here is an example value for this property: c:\app\foo.dll,c:\anotherapp\bar.dll
file_buffer_size
The size of the buffer to read and write static files to the Socket connected to the client.
virtual_directories_config
The config file that defines virtual directories. This is a text file with alternating lines of physdir:[physical path] and virtpath:[virtual path], where the [physical path] can be a fully qualified pathname to an existing physical directory or a relative path and the [virtual path] is a URI path starting with the / character that is to be mapped to the physical directory. The virtpath: MUST follow a physdir:. An example of a virtual directory config file:

                      physdir:C:\Projects\Web App Projects\Calendar App
                      virtpath:/apps/calendar
                      physdir:..\..\Users\Docs
                      virtpath:/mydocs
                      physdir:\\mynetworkDrive\share\html
                      virtpath:/share
                    
restricted_directories
A set of directory names that will be denied access if they appear in the URI path. They are separated by commas and must not include any / path separators.
restricted_access_404
If this property is set to true, attempted accesses to restricted directories will return a 404 Not Found instead of the default 403 Forbidden. Set this to true if you want the added protection of less information disclosure of making access to a restricted directory appear as if the directory isn't there at all.

For virtual hosts (serving different websites with unique domain names a.k.a. multihoming), start multiple HTTPFileServerAgent instances and give each one unique service_name and html_directory prop values. Include the domain name (or the protocol, http:// or https://) with the URI in the agent's service_name. Here are some examples of valid service_names:


                http://www.server.org/
                http://www.server.org:8080/
                https://www.server.org/
                www.server.org/
                www.server.org:8080/
                /
              

HTTPProxyAgent

HTTPProxyAgent handles proxies to other web servers. It can provide complete http proxy services to browsers and other clients connecting on the addresses and ports specified in the listeners property, and it can proxy specific URLs from the http.proxy file while the WebServer handles other requests normally.

The http.proxy file lists full URLs or URIs to proxy and where they should be proxied to if the WebServer is running. It consists of name/value pairs separated by newlines. Lines beginning with the # character are ignored as comments. The value of each name/value pair must be a complete URL.

Here are some valid name/value pairs that might be used in the http.proxy file:

http://myserver/proxy_me=http://proxy:1099
http://myserver/proxy_me2=http://proxy1:9000
http://myserver/proxy_me2=http://proxy2:9000
http://myserver/proxy_me2=http://proxy3:9000
When a browser requests /proxy_me, the HTTPProxyAgent contacts the web server at proxy on port 1099 and proxies the request to that server. When a browser requests /proxy_me_too, the HTTPProxyAgent cycles through the 3 web servers indicated, so that the first time the URL is accessed, the connection is proxied to proxy1:9000. The second time, the request is proxied to proxy2:9000, the third request goes to proxy3:9000, the fourth request goes to proxy1:9000, and so on. Both the name and the value of each name/value pair are assumed to be literal, so they will only match the exact path/hostname of the http request.

The servers listed in the http.proxy file are periodically checked to ensure that they are up and responding. The HTTPProxyAgent will not proxy requests to hosts from the http.proxy file that aren't responding.

HTTPProxyAgent also supports listeners, which operate independently of the WebServer to proxy traffic for browsers configured to use a proxy server. Any number of listeners can be started to provide proxy services on specific ports or IPs. Listeners are specified using the listeners prop of the HTTPProxyAgent.

This agent supports the following Props entries:
service_name
The name this agent uses to register with Service Manager. The WebServer's proxy_agent prop must be set to match this agent's service_name, or requests for urls in the http.proxy file will not be handed off to this agent. The default value is http_proxy.agent.
proxy_file
The path to a file containing a list of the available proxies. See above for syntax. Defaults to ./http.proxy
log_requests
When this prop is set to true, the agent will log requests handled by proxy listeners. Requests routed through the WebServer are logged in the WebServer's http logs. Defaults to true.
proxy_log_dir
The path to a directory containing proxy log files. Defaults to .\proxy_logs.
check_interval
Specifies the time to wait (in millisecs) before checking a proxy's status. Defaults to 300000
initial_wait_time
Specifies the time to wait after startup (in millisecs) before checking a proxy's status. Defaults to 600000
listeners
A comma separated list of hostnames or IP addresses (with port number) on which to provide standard HTTP proxy services for browsers. Requests on these ports won't be routed through the WebServer. By default, this prop is not set.
snoop_traffic
A boolean value that determines if data sent through the http proxy will be output to the console or file logger. By default, this prop is false.
HTTPPutAgent This agent can be loaded to support traditional HTTP Put functionality, saving uploaded files to the directory specified in the html_directory prop. To run this agent, the WebServer must have a prop called http_put.agent that matches the HTTPPutAgent's service_name.
service_name
The name this agent uses to register with Service Manager. The default value is http_put.agent.
html_directory
The path to the directory where received files will be stored. Defaults to ./
make_directories
If this prop is set to true, the HTTPPutAgent will create new sub-directories under the server's html_directory when a client uploads a file to a non-existent directory. Defaults to false
HTTPRedirectAgent

This Request Agent is used by the WebServer to determine whether a web request should be redirected and to actually redirect the request by returning the proper redirect code to the client.

If an HTTPRedirectAgent is running and the WebServer's redirect_agent prop is the same as the HTTPRedirectAgent's service_name, the agent will redirect requests for URLs in the http.redirect file. This file lists where to redirect to; URIs with multiple redirects listed in the file will be cycled between the listed redirects, providing a round-robin load distribution scheme.

The file format is name/value pairs delimited by newlines; lines starting with '#' are ignored. Complete URLs must be used to specify redirected URLs and the redirect targets. Here is an example of a redirect file:

http://myserver/redir1/file.htm=http://yahoo.com/
http://myserver/redir2/=http://redirect1:9000/
http://myserver/redir2/==http://redirect2:9000/
http://myserver/redir2/==http://redirect3:9000/
When a browser requests /redir1/file.html, it is redirected to http://yahoo.com. When it requests /redir2, it is redirected to http://redirect1:9000/. The next request for /redir2 is redirected to http://redirect2:9000/. The third incoming request is redirected to http://redirect3:9000/. The fourth request is redirected to http://redirect1:9000/, and the cycle is started again. In this way, the HTTPRedirectAgent can be used for round-robin style load balancing.

There are 2 kinds of name/value pairs used in the redirect configuration file. The first is a direct mapping in the format NAME=VALUE. This mapping tells the redirect agent to redirect PATH NAME to PATH VALUE.

A directly mapped NAME that contains a hostname must have a host in its VALUE. A direct mapping will not recurse subdirectories in a path and each path is regarded as literal. Directly mapped names can by mapped to multiple values if the values are separated by commas. The agent will cycle through each of the different paths given as the value.

The second kind of mapping maps only hosts to hosts, and passes the requested URL along with the redirected request for the mapped host to handle. This is an example of host-to-host mapping syntax:

http://myhost.com>http://www.myhost.com
No path information is included in the syntax of this mapping, only the host names. Any path intended for myhost.com will be mapped to the host www.myhost.com.

Note: Cyclical references in the redirect mappings are not checked, if the redirect file has cyclical references, clients can be put into an infinite loop of redirects.

service_name
The name this agent uses to register with Service Manager. The WebServer's proxy_agent prop must be set to match this agent's service_name, or requests for urls in the http.redirect file will not be handed off to this agent. The default value is http_redirect.agent.
redirect_file
The path to the file where redirect mappings are specified. Defaults to ./http.redirect
check_interval
Specifies the time to wait (in millisecs) between checking to see if the redirect target servers are up. Defaults to 300000
initial_wait_time
Specifies the time to wait after startup (in millisecs) before checking to see if the redirect target servers are up. Defaults to 600000
check_hosts
If set to true, the HTTPRedirectAgent will check every check_interval milliseconds to ensure that servers handling redirects are up. If one goes down, requests are not redirected to that server until it comes back online. Defaults to true
default_port
Specifies the port to redirect requests on. Defaults to 80
HTTPRequest This class is used as a data object which stores the HTTP/1.0 request. It will provide the HTTP/1.0 request in String form.
HTTPResponse This class is a data object which stores a HTTP/1.1 response.
KTcpClient This class extends TcpClient to provide access to the underlying socket and IP Address of the remote machine.
SecureWebServer SecureWebServer extends WebServer with SSL security features. You must provide a certificate suitable for server authentication, and any passwords or key files needed to access the certificate. SSL3 and TLS3 are the only supported protocols. This agent supports the following Props entries:
socket_linger_time
The number of millisecs to wait after an http request is completed before closing the socket used for that request. Default value is 2000
bind_port
The port that the webserver will bind to. Default value is 80
max_connections
The maximum number of concurrent connections to allow. Default value is 20
index_agent
The agent that gets called when the default page is requested. Default value is HTTPFileServerAgent
bind_ip
The IP address to service HTTP requests on (on mulithoming machines. To service requests on any IP interfaces, use 0.0.0.0 as the value. Default value is 0.0.0.0
log_requests
If this is set to true, the server will create HTTP access logs. Default value is true
log_file_dir
Specifies the directory where the server will create the HTTP access logs. Default value is ./logs
mime_types_file
The path to a file specifying mime types to use with various file extensions. If this file is missing, the server uses common default mappings. Default value is ./mime_types
server_certificate
The path to the certificate this server will use for SSL connections. To retrieve a certificate from the local host's certificate store, use the syntax store:/MY/localhost. Note that the example above doesn’t include a certificate name. If no certificate name is provided the server will use the 1st certificate it finds in the specified certificate store that is suitable for SSL server authentication. Use the following values to access various certificate stores on windows platforms:
CA
Certification authority certificates.
MY
A certificate store holding "My" certificates.
ROOT
Root certificates.
SPC
Software publisher certificates.
Default value is ./test-cert.pfx
certificate_password
The password to use when accessing the certificate. (The password for the certificate stored in the file test-cert.pfx is abcd, so the default value is abcd
.
The following Props settings are optional:
server_key_file
The path to a server key file. By default this prop is not set.
key_file_password
The password used to access the server key file. By default this prop is not set.
WebServer

The Neokernel web server services http requests. If an incoming request is for a static file like an HTML file, the web server delivers it. If the request URI ends in .agent, the web server uses a custom C# or Visual Basic object to generate the response. Objects that generate responses in this way implement the RequestAgent interface. If the server can't find a file or a RequestAgent to match the client's request, a 404 (file not found) error is returned. Typically, an HTTPFileServerAgent is used to serve static files like images or HTML. The HTTPFileServerAgent also handles ASP.NET pages if the file ends with the .aspx extension.

Custom RequestAgents are commonly used in conjunction with the WebServer and SecureWebServer classes to handle HTTP requests dynamically. Here's an example of a simple RequestAgent written in Visual Basic:

   Imports com.neokernel.nk
   Imports com.neokernel.util

   Public Class SimpleVBPage
      Inherits RequestAgent

      Overrides Sub handleRequest(ByVal Request As AgentRequest)
         Dim title, body As String
         body = getString("body")
         Request.println(body)
      End Sub

      Overrides Sub initProps()
         setDefault("service_name", "/simple.agent")
         setDefault("body", "This is output from the SimplePage agent!")
      End Sub
   
   End Class
 

If a browser requests http://localhost/simple.agent from a server that has started the above SimpleAgent, the browser will display the text

This is output from the SimplePage agent!

Here is the exact same agent written using C#:

   using System;
   using com.neokernel.nk;

   public class SimplePage : RequestAgent
   {
      public override void initProps()
      {
         setDefault("service_name", "/simple.agent");

         setDefault("body", "This is output from the SimplePage agent!");
      }

      public override void handleRequest(HTTPAgentRequest request)
      {
         String body = getString("body");

         request.println( body );
      }
   }
 

WebServer supports the following Props entries:

socket_linger_time
The number of millisecs to wait after an http request is completed before closing the socket used for that request. Default value is 2000
bind_port
The port that the webserver will bind to. Default value is 80
max_connections
The maximum number of concurrent connections to allow. Default value is 20
index_agent
The agent that gets called when the default page is requested. Default value is HTTPFileServerAgent
bind_ip
The IP address to service HTTP requests on (on mulithoming machines.) To service requests on any IP interfaces, use 0.0.0.0 as the value. Default value is 0.0.0.0
log_requests
If this is set to true, the server will create HTTP access logs. Default value is true
log_file_dir
Specifies the directory where the server will create the HTTP access logs. Default value is ./logs
mime_types_file
The path to a file specifying mime types to use with various file extensions. If this file is missing, the server uses common default mappings. Default value is ./mime_types
max_uri_length
The maximum length (in bytes) of URIs that the server will accept before a request is ignored. The default value is 4096
max_multipart_length
The maximum size of a POST request that the server will accept (in bytes) before the request is ignored. Default value is 4194304 (4MB)
refuse_requests
This prop specifies a list of strings that will result in a request being refused if a client passes them in the request URI. By default this prop is not set.
multihome
If this prop is set to false, multihome functionality is disabled. By default this prop is set to
true
redirect_agent
If this prop is set to true, the server will use the HTTPRedirectAgent to redirect incoming requests. This prop is not set to a default value.
proxy_agent
If this prop is set to true, the server will use the HTTPProxyAgent to proxy incoming requests. This prop is not set to a default value.
authentication_agent
If this prop is set to true, the server will use the HTTPAuthenticationAgent to demand passwords for certain URLs specified in the http.authentication file. This prop is not set to a default value.

XHTMLFormWriter XHTMLFormWriter is a tool for writing simple XHTML-based forms to a StringBuilder or AgentRequest output stream without worrying about the XHTML code.

Interfaces

Interface Description
HTMLResponseInterface This interface defines an Object that can return the full HTTP response header and body (as HTML), based off of the constants in com.neokernel.httpd.WebServer