Neokernel User Guide Release 2.0

3. This mono-spaced font is used to denote examples of source code and the names of classes or namespaces, unless the name is being used in a section title.

4. Italics indicate emphasis.

5. Meta-names (for example, words like YourName), which describe the type of text to be entered rather than the literal string, are enclosed in angle brackets, e.g.: <YourName>

Part One: Administrators Guide

Introduction

The Neokernel is a simple, lightweight, secure, and high performance web server. It can be embedded and distributed with other applications or used immediately as a production web server. In addition to serving static files, the server can execute custom C# and Visual Basic software components to generate dynamic content in response to web requests.

The Neokernel lets your application serve static and dynamic web pages without relying on a pre-installed or external web server.

Starting the Server

The Neokernel can be started in three ways:

1. Using the neokernel.exe file. The executable can be launched using Windows Explorer, from a command prompt, or with a .bat file.

2. Running the server as a Service. If the Neokernel Windows Service component is installed, the server can be started and stopped from the Services control panel on Windows NT, 2000 and XP.

3. By embedding the server library in your own .NET application and using the provided APIs to start the server process.

The Neokernel must have a valid license key file, named license, located in the same directory as the executable or library file. The Neokernel comes with a free 30 day license. License keys are available from http://www.neokernel.com.

Command Line Arguments

The neokernel.exe executable supports the following command line syntax:

neokernel [-hide_println] [-hide_debug] [-hide_warning] [-startup_dir <dir>] [-startup_file <.xml startup file>]

-startup_dir <dir>	Where <dir> is the path to the directory that neokernel.exe should use as a startup directory.
-startup_file <file>	Where <file> is the path to an xml props file that neokernel.exe should use as the startup file.
hide_println	If this prop is set to true, all println() output from this kernel is hidden. The default value is false.
hide_debug	If this prop is set to true, all debug output from this kernel is hidden. The default value is false.
hide_warning	If this prop is set to true, all warning output for this kernel is hidden. The default value is false.

Table 1 – Command line arguments supported by neokernel.exe.

Configuring Server Properties ( Props)

Most server behavior is configurable by editing the neokernel_props.xml file. This file is divided into sections corresponding to the components that the server will load at startup.

Each section defines a set of properties (Props[1]) used to configure a specific server component. Neokernel Server components are called agents and include agents for serving web pages and images, authentication, proxy services, and redirection.

If you define a value for a prop in the neokernel_props.xml file, the agent will use that value instead of a default value. Default values are used when the prop isn’t specified. The /demos directory has examples of various server configurations with different neokernel_props.xml files that can be used as templates for your own server configurations.

Multiple Neokernel Server processes can be running at the same time, each with a unique neokernel_props.xml file. The properties in the file are read when the Neokernel Server is started; if you edit the file, you must re-start the Neokernel before your changes will take effect.

The startup file doesn’t have to be called neokernel_props.xml if the server is started with the -startup_file argument to specify a file with a different name.

Note: If you specify a startup file in a different directory than the Neokernel executable, the -startup_dir does not change and the Neokernel’s -startup_dir is the executable directory. Unless the -startup_dir is specified as well, relative paths in the neokernel_props.xml file will be resolved relative to the executable’s directory and not the directory with the startup file.

Other files are also used for configuring the Neokernel Server. The following table describes each of these files[2].

File Name	Purpose
neokernel_props.xml	An XML server configuration file with the properties for agents that the server will load. Here is an example of a very simple neokernel_props.xml file: <?xml version="1.0"?> <props_list> <props> <agent_id>100</agent_id> <classname>AgentOne</classname> <startup>true</startup> </props> <props> <agent_id>101</agent_id> <classname>AgentTwo</classname> <startup>true</startup> </props> </props_list> </xml>
authmap	If the HTTPAuthenticationAgent is running, it uses this file to determine what directories on the server require authentication. The file must contain the names of each secured directory, or realm, and the username / password combinations that are allowed to access that realm. The authentication map file must be in this format: default PATH0 REALM0 username0 password0 PATH1 REALM2 username0 password0 username1 password1 Here is an example of an authentication file: default /private/mine/ My Realm john johnpassword santos halper /private/ Main Realm john mainpassword If the above example is used as the authentication map file, attempts to access http://hostname/private/mine/ will be prompted for a username and password (the prompt will ask for the password to My Realm). If you want to password protect an agent-generated page, set the agent’s service_name so it includes the protected realm (directory) in it’s URL. The default keyword at the start of the example means that the realms apply to ALL domains that this server handles. If you want a realm to apply only to a certain domain, replace the default keyword with the hostname of the domain you want. Multiple domains may be listed in a single authentication map file, as in the following example: www.myserver.com /private/mine/ My Realm john johnpassword santos halper www.myotherserver.com /private/ Main Realm john mainpassword
redirect	If the HTTPRedirectAgent is running, it uses this file to determine what requests should be redirected and where they should be redirected to. Here is an example of a redirect file: http://myserver/redir1/file.htm=http://yahoo.com/ http://myserver/redir2/=http://redirect1:9000/ http://myserver/redir2/==http://redirect2:9000/ http://myserver/redir2/==http://redirect3:9000/ When a browser requests /redir1/file.html, it is redirected to http://yahoo.com. When it requests /redir2, it is redirected to http://redirect1:9000/. The next request for /redir2 is redirected to http://redirect2:9000/. The third incoming request is redirected to http://redirect3:9000/. The fourth request is redirected to http://redirect1:9000/, and the cycle is started again. In this way, the HTTPRedirectAgent can be used for round-robin style load balancing. Complete URLs must be used to specify redirected URLs and the redirect targets.
proxy_hosts	If the HTTPProxyAgent is in use, this file provides the list of URLs that will be proxied instead of being handled normally by the Web Server. Here is an example of a proxy file: http://myserver/proxy_me=http://proxy:1099 http://myserver/proxy_me2=http://proxy1:9000 http://myserver/proxy_me2=http://proxy2:9000 http://myserver/proxy_me2=http://proxy3:9000 When a browser requests /proxy_me, the WebServer gives the request to the HTTPProxyAgent to handle. The HTTPProxyAgent contacts the web server at proxy on port 1099 and proxies the request to that server. When a browser requests /proxy_me_too, the HTTPProxyAgent will cycle through the 3 web servers indicated, so that the first time the URL is accessed, the connection is proxied to proxy1, port 9000. The second time, the request is proxied to proxy2, the third request goes to proxy3, the fourth request goes to proxy1, and so on. The HTTPProxyAgent also supports listeners, which can proxy all HTTP or HTTPS traffic instead of specific URLs. Listeners are used when you want to provide a proxy for HTTP browsers like Internet Explorer; they are configured to listen on a specific port and IP number, and are specified using the listener prop of the HTTPProxyAgent. For more information, see page 15.
mime_types	This file describes what MIME types the server should use when sending browsers a file with a specific file extension. If a mime_types file isn’t found, a default set of common MIME types is used.
log_maps	The LogViewerPage provides a web interface for viewing summary information from the web server’s log files. If it is running, the log_maps file defines the reports that the LogViewerPage generates. This file should contain names mapped to text patterns. For example, a map file might look like this: Total requests=GET LogViewer requests=GET /logviewer.agent In the above example, LogViewerPage will let the user select from 2 fields, Total requests and LogViewer requests. If the user chooses the report called Total requests, all log entries that match the pattern ‘GET’ will be counted and included in the report. If the user chooses to view the report called LogViewer requests, a smaller set of entries (that match the pattern ‘GET /logviewer.agent’) will be included in the second report. The search patterns used to filter the reports can be regular expressions.

Table 2 – Server configuration files.

Server Logs

The server generates log files that record server processes and client requests. By default, HTTP log files are generated in a subdirectory of the startup directory called /logs. This directory will be created if it doesn’t exist. The http log files conform to the NCSA-Extended Log File formatting conventions.

By default, all other program output gets routed to the console. If you need to store the console output in log files, look in the neokernel_props.xml file for the com.neokernel.io.FileReporter section and set the startup prop of this agent to true. This agent will log all console output to sequentially numbered text files in the /logs subdirectory of the server’s working directory. You can change the log directory by setting the log_dir prop to a different path (if the log_dir prop is not set, the default value is ./logs).

Running the Neokernel as a Windows Service

The nkservice.exe executable is for running the Neokernel as a Windows Service. The installer registers this executable as a windows service so it can be controlled using the Services control panel. The Neokernel service supports the same command line arguments as the Neokernel console executable.

When the Neokernel is running as a windows service, all of the output that normally appears in the console is logged to sequentially numbered files in the /logs subdirectory of the startup directory. Errors and other significant events will also appear in the Windows Event log, viewable using the Event Viewer[3].

How it Works – Just the Basics

The Neokernel works like other web servers; if a browser requests a file from the server and the file is available, the server delivers it. However, if the requested URL ends in .agent, the server looks for a C# or Visual Basic object mapped to the URL and uses it to generate the response. Classes that handle requests for the server extend the class RequestAgent. The WebServer class hands off incoming requests to these RequestAgent[4] objects which generate dynamic content when their URL is requested. If no file or RequestAgent is found to match the incoming request, a 404 (file not found) error is returned. The WebServer class also manages logging, error responses, and determining the MIME types of files being requested[5].

RequestAgent objects handle requests for the URL specified in their service_name prop. Therefore, when a client requests http://myserver/somepage.agent, the WebServer searches all loaded RequestAgents for an agent with a service_name prop set to /somepage.agent. If it finds one, it calls that agent’s handleRequest method and the agent generates a response.

Here’s a step by step walkthrough:

6. A client submits a request and the server creates an HTTPRequestObject from the client’s request and the open socket connection with the client.

7. The server hands off the HTTPRequestObject to an agent specified by the requested URL.

8. The agent responds to the request using the print() and println() methods of the HTTPRequestObject to send data to the client.

Figure 1 – Neokernel web request task flow.

Virtual Hosts

The Neokernel supports virtual hosts[6], which refers to serving different domains from a single web server. Usually each site is mapped to a different server directory with content for that domain. Visitors to these sites will be routed to the correct virtual host, with its own default pages and web links.

Agents that handle requests for a specific virtual host must include the domain in their service_name when running in a virtual hosts configuration.

To use virtual hosts:

9. Set the virtual_hosts prop of the WebServer to true and create directories with the content for each of the virtual sites.

10. Create HTTPFileServerAgent entries in the neokernel_props.xml file for each directory from step 2.

11. Set the service_name prop of each HTTPFileServerAgent to the virtual host name (e.g. domain name) that it should handle requests for, and set their html_directory props to the path of the directory with their site content, as in the following example:

Here are examples of valid service_name settings:

http://www.server.org/homepage.agent

http://www.server.org:8080/homepage.agent

https://www.server.org/homepage.agent

www.newserver.org/homepage.agent

www.newserver.org:8080/homepage.agent

/homepage.agent

To review, when the web server gets a request, it tries to find an agent whose service_name prop matches the requested URL. If virtual hosts is active, the server first looks for an agent with the hostname that the client is using appended in the beginning. If no matching agents are found, the server tries to load the URL without a hostname.

When the domain is not included as part of an agent’s service_name prop, that agent will respond to requests that match its URL regardless of the hostname requested by the client.

For example, if an agent registers with the service_name of /homepage.agent, and the host is registered in the DNS as both www.server.org and www.newserver.org, then the agent will handle requests for both http://www.newserver.org/homepage.agent and http://www.server.org/homepage.agent.

However, if the service_name includes the server’s hostname, for example http://www.newserver.org/homepage.agent, the agent will respond to requests for http://www.newserver.org/homepage.agent, but not to requests for http://www.server.org/homepage.agent. Requests for the second URL will result in a 404 (file not found) error.

Props Supported by All Agents

The following props are supported by all agents. If no entry for a prop is specified in the neokernel_props.xml file, a default value for that prop is used. Agents that interact with other agents typically have a service_name property in addition to those listed here:

name	The agent’s display name; this will be used in agent output to identify messages from this agent. Defaults to be the same as the classname
startup	This boolean prop determines if this agent will be started by the server. The default value is true
id	The agent's id (a numeric value.) The kernel loads agents in numeric order according to their agent id. The default is for agents to use an id assigned by the kernel.
classname	The name of the object that contains this agent’s code.
hide_println	If this prop is set to true, all println() output for this agent is hidden. The default value is false
hide_debug	If this prop is set to true, all debug output for this agent is hidden. The default value is false
hide_warning	If this prop is set to true, all warning output for this agent is hidden. The default value is false

Table 3 – Props supported by all agents.

com.neokernel.httpd.WebServer

Multiple WebServer agents can run within a single Neokernel process, but each must be running on a unique port (as specified by it’s bind_port prop).

bind_port

This prop tells the web server what port to bind to and accept connections from.

bind_ip

This prop specifies the IP address that the server will bind to. To bind to every IP address on the host machine, set this prop to 0.0.0.0

virtual_hosts

Set this prop to true to enable support for virtual hosts.

max_connections

This prop determines how many simultaneous connections the web server will handle. Any connection made after this number of connection has been reached will hang until the next thread becomes available to handle it, or for the amount of time specified in thread_pool_timeout.

index_agent

This prop specifies the agent that handles requests for the server’s root or default directory (e.g. http://www.neokernel.com/). Typically this is set to an HTTPFileServerAgent which serves a default_index page.

log_file_dir

This prop defines the path to the directory where the server will output NCSA-Extended format http log files. By default this prop is set to ./logs

log_requests

If this prop is set to true, the WebServer will log http requests to sequentially numbered text files in the directory specified in log_file_dir.

mime_types_file

This prop contains the path to a file describing what mime types to use for specific file extensions; this file is also mentioned on page 5. By default this prop is set to ./mime_types

refuse_requests

This prop specifies a list of strings that will result in a request being refused if a client passes them in the request header. This can be used to block access based on the IP or “user-agent” description passed with the request. For example, the value

204.182.54.1,Googlebot,Windows ME

tells the server to refuse requests from the IP address of 204.182.54.1, the Google indexer bot, and anyone using Windows ME. By default this prop is not set.

socket_linger_time

The number of milliseconds the agent should wait after an http request is completed before closing the socket used to service that request. The default value is 2000

proxy_agent

If this prop is set, the WebServer will use it’s value as the service name to get a reference to the HTTPProxyAgent from the ServiceManager. The HTTPProxyAgent will be used proxy incoming requests for any of the URIs in the http.proxy file, and for http proxy traffic on the IP addresses and ports specified in the listeners prop of the HTTPProxyAgent. Set the value of this prop to http_proxy.agent or to the service_name of an HTTPProxyAgent running in your server if you have configured it to use a different service name. This prop is not set by default.

redirect_agent

If this prop is set, the HTTPRedirectAgent will be used to redirect requests for the URIs specified in the http.redirect file. Set the value of this prop to http_redirect.agent or to the service_name of an HTTPRedirectAgent running in your server. This prop is not set by default.

authentication_agent

If this prop is set, the HTTPAuthenticationAgent will be used to authenticate incoming requests. Set the value of this prop to http_authentication.agent or to the service_name of an HTTPAuthenticationAgent running in your server. This prop is not set by default.

Table 4 - com.neokernel.httpd.WebServer

com.neokernel.httpd.SecureWebServer

SecureWebServer supports secure communications using SSL. It supports the same props as WebServer, and adds support for the following:

server_certificate

This prop tells the server where to find a certificate for SSL communications.

If the certificate is a file on the disk, use the path to that file as the value of this prop. For instance:

c:\\neokernel\\localhost-cert.pfx

If the certificate is installed in one of the host machine’s certificate stores, use the syntax

store:[name of certificate store]

The value for [name of certificate store] will usually be MY meaning the group of certificates listed under the Personal tab in the Certificates window[7]. A typical setting for this prop when loading certificates from the local certificate store is:

store:MY

The server will use the first cert in the specified store that is appropriate for SSL server use. Use the following values to access the various certificate stores on windows platforms:

CA	Certification authority certificates.
MY	A certificate store holding "My" certificates.
ROOT	Root certificates.
SPC	Software publisher certificates.

The default value of this prop is localhost-cert.pfx, which is the name of the example certificate provided for testing SSL connectivity.

certificate_password

The password to use when accessing the SSL certificate. Note: The password for the example certificate provided in the file localhost-cert.pfx[8] is abcd.

The following props may or may not be necessary depending on the type of certificate you are using:

server_key_file

The private key file for the server certificate. By default this prop is not set because the example certificate

(localhost-cert.pfx) has a private key embedded in the certificate.

key_file_password

This is the password for accessing the .pvk file specified in the key_file prop. By default this prop is not set.

Table 5 - com.neokernel.httpd.SecureWebServer

com.neokernel.httpd.HTTPFileServerAgent

This agent serves html, ASP.NET, image, and other files from a specified directory. To serve different web sites from a single WebServer, start multiple instances of FileServerAgent[9] and give each a unique service_name prop matching the domain or hostname that it should handle requests for. The \Virtual Hosts example in the \demos directory shows how to do this.

Here are some examples of valid service_name entries in a virtual hosts configuration:

http://www.server.org/

http://www.server.org:8080/

https://www.server.org/

www.server.org/

www.server.org:8080/

If the requested filename ends with .aspx, the file will be passed to the ASP.NET Runtime and code contained in the file will be executed before the content is returned to the requestor. If the ASP.NET code references other assemblies or .dlls, copies of these must be in the same folder (or the \bin subdirectory of the folder) where the ASP.NET (.aspx) file is located.

html_directory	This prop specifies the directory containing the static files that will be served by this agent. By default, this prop is set to ./html
default_index	This prop specifies what (if any) file to serve when the URL requested by a client is a directory name. This prop is set to index.html by default, which means if a browser requests /public_html/my_stuff/, the server will look for the file at /public_html/my_stuff/index.html.
aspnet_config_file	The path to an XML file containing ASP.NET configuration and debugging information. Defaults to web.config.
preload_aspx_pages	A list describing the ASP.NET pages that should be pre-loaded when the agent starts. This is not required but it makes pages load faster the first time they are requested by the user. To pre-load ASP.NET pages the agent must know the file location and it's URL relative to the server root. The list of files is delimited by semi-colons and looks like: [WebPath1],[FilePath1];[WebPath2],[FilePath2] …and so on, where the first item in each pair is the path to the file relative to the root of the web server (e.g. /aspapp/myfile.aspx) and the second is the path to the file on the hard disk (e.g. d:\htmldir\aspapp\myfile.aspx). This property is not set by default.
aspx_assemblies	An optional comma seperated list of assemblies referenced by ASP.Net pages. Assemblies appearing in this list will be copied to the \bin subdirectory of the folder where the ASP.Net page is located. The files will be re-copied and re-loaded if a newer version becomes available. Use the full path. Here is an example value for this property: c:\app\foo.dll,c:\anotherapp\bar.dll
allow_directory_lists	By default this prop is set to no. If this prop is set to yes the server will dynamically construct an HTML document which lists the files in the requested directory (if it is a subdirectory of html_directory and there is no default_index file) and return it to the browser.

Table 6 - com.neokernel.httpd.HTTPFileServerAgent

com.neokernel.httpd.HTTPAuthenticationAgent

This agent handles HTTP authentication. A configuration file defines the realms[10] that require authentication and the username/password pairs to use when authenticating with each realm. HTTPAuthenticationAgent is used by the WebServer to handle authorization of password protected URIs.

The authentication file defines the realms and the list of username/password combinations that can be used to authenticate with each realm. A “realm” is the name of the protected item or website; this will be shown to the user when they are asked to authenticate. The authentication file is delimited by newline characters and must end with a newline. Each realm is declared with a PATH and the REALM name separated by spaces, followed by usernames and passwords on new lines, separated by a space. PATHs must begin with a slash and end with a slash.

The HOSTNAME field at the beginning of the file indicates to the HTTPAuthentication agent what hostname the realms should be mapped to. The default hostname is default, meaning the http.authentication file applies to all hostnames on the server machine.

Here is an example of an authentication file for a website that will ask for authentication if the user requests anything from the folders /PATH0 or /PATH2:

        HOSTNAME

        PATH0 REALM_NAME0

        username0 password0

        username1 password1

        username2 password2

        username3 password3

        PATH2 REALM_NAME0

        username0 password0

        username1 password1

The next example shows an authentication file with two realms, 'My realm' and 'Main Realm'. If any access is made to files or directories in /private, the username jonlin with the password mainpassword will be expected, except for accesses to /private/mine/, which is under the /private path but is its own realm and requires its own usernames and passwords. If default is used as the hostname then the authentication map will apply to all virtual hosts on this server.

Here is an example of an authentication map file that applies to every hostname:

        default

        /private/mine/ My realm

        john johnpassword

        santos halper

        /private/ Main Realm

        john mainpassword

service_name

The name this agent uses to register with the ServiceManager. The WebServer’s authentication_agent prop must be set to a matching value so that requests for URLs requiring authentication are handed off to this agent. The default value is http_authentication.agent

authentication_files

This prop holds the path to the authentication map file (or multiple files in a comma separated list) described on page 5. The default value is .\http.authentication

Table 7 - com.neokernel.httpd.HTTPAuthenticationAgent

com.neokernel.httpd.HTTPProxyAgent

This agent serves as a proxy for http and https traffic. It can provide complete proxy services to browsers and other clients connecting on the addresses and ports specified in the listeners property, and it can proxy the specific URLs listed in the http.proxy file while the WebServer handles other requests normally.

The http.proxy file lists full URLs or URIs to proxy and what server to get the data from (if it is running.) It consists of name/value pairs separated by the newline character. Lines beginning with the # character are ignored as comments. The value of each name/value pair must be a complete URL.

Here are some valid name/value pairs that might be used in the http.proxy file:

        http://myserver/proxy_me=http://proxy:1099

        http://myserver/proxy_me2=http://proxy1:9000

        http://myserver/proxy_me2=http://proxy2:9000

        http://myserver/proxy_me2=http://proxy3:9000

When a browser requests /proxy_me, the HTTPProxyAgent contacts the web server at proxy on port 1099 and proxies the request to that server. When a browser requests /proxy_me_too, the HTTPProxyAgent cycles through the 3 web servers indicated, so that the first time the URL is accessed, the connection is proxied to proxy1:9000. The second time, the request is proxied to proxy2:9000, the third request goes to proxy3:9000, the fourth request goes to proxy1:9000, and so on. Both the name and the value of each name/value pair are assumed to be literal, so they will only match the exact path/hostname of the http request.

The servers listed in the http.proxy file are periodically checked to ensure that they are up and responding. The HTTPProxyAgent will not proxy requests to servers in the http.proxy file that aren't responding.

HTTPProxyAgent also supports listeners, which operate independently of the WebServer to proxy traffic for browsers configured to use a proxy server. Any number of listeners can be started to provide proxy services on specific ports or IPs. Listeners are specified using the listeners prop of the HTTPProxyAgent.

service_name	The name this agent uses to register with the ServiceManager. The WebServer’s proxy_agent prop must be set to a matching value so that requests for proxied URLs are handed off to this agent. The default value is http_proxy.agent
proxy_file	This prop holds the path to the proxy file described on page 5. By default, this prop is set to ./http.proxy
check_interval	The HTTPProxyAgent has an internal check where it pings the web servers on the proxy list. If they are not responding, that web server will be removed from the proxy list temporarily. It will be added again once the server is responding. By default, this is set to 300000, which means every 300000 milliseconds the agent will check each proxy destination on its list.
listeners	This is a comma separated list of IP addresses with port numbers on which the agent will proxy ANY http or https request. This is used for proxying all requests as opposed to when only certain URLs are proxied. A typcial setting for this prop is 0.0.0.0:80,0.0.0.0:443, which means the agent will provide HTTP proxy services on all local IP addresses on port 80, and on all local IP addresses on port 443 (the default port for https traffic). Note that this setting will conflict with the default port that the WebServer runs on, so if you use proxy listeners on the same machine as a WebServer you’ll need to use different ports. By default this prop is not set.
log_requests	When this prop is set to true, the agent will log requests handled by proxy listeners. Requests routed through the WebServer are logged in the WebServer’s http logs
proxy_log_dir	The path to a directory where proxy log files will be written to. Defaults to .\proxy_logs
initial_wait_time	This prop specifies how long to wait after startup before starting to ping the proxy destinations. By default this prop is set to 60000, which means the agent will wait 60000 milliseconds before it starts checking the servers in the list.

Table 8 - com.neokernel.httpd.HTTPProxyAgent

com.neokernel.httpd.HTTPRedirectAgent

This agent is used by the WebServer to determine whether a web request should be redirected and to redirect requests that should be redirected.

If an HTTPRedirectAgent is running and the WebServer's redirect_agent prop is the same as the HTTPRedirectAgent's service_name, the agent will redirect requests for URLs listed in the http.redirect file. This file lists where to redirect to; URIs with multiple redirects listed in the file will be cycled between the listed redirects, providing a round-robin load distribution scheme.

The file format is name/value pairs delimited by newline characters; lines starting with # are ignored. Complete URLs must be used to specify redirected URLs and the redirect targets. Here is an example of a redirect file:

       http://myserver/redir1/file.htm=http://yahoo.com/

       http://myserver/redir2/=http://redirect1:9000/

       http://myserver/redir2/=http://redirect2:9000/

       http://myserver/redir2/=http://redirect3:9000/

There are 2 kinds of name/value pairs used in the http.redirect file. The first is a direct mapping in the format NAME=VALUE. This mapping tells the redirect agent to redirect PATH NAME to PATH VALUE.

A directly mapped NAME that contains a hostname must have a host in its VALUE. A direct mapping will not recurse subdirectories in a path and each path is treated as literal. Names can by mapped to multiple values if the values are separated by commas. The agent will cycle through each of the different paths given as the value.

The second kind of mapping maps only hosts to hosts, and passes the requested URL along with the redirected request for the mapped host to handle. This is an example of host-to-host mapping syntax:

        http://myhost.com>http://www.myhost.com

No path information is included in the syntax of this mapping, only the host names. Requests for myhost.com will be mapped to the host www.myhost.com. This mapping is especially useful when the WebServer is running virtual hosts, because multiple hostnames can all by mapped to a single host or URL.

Note: Cyclical references in the redirect mappings are not checked, if the redirect file has cyclical references, clients can be put into an infinite loop of redirects.

redirect_file	This prop determines the path to the redirect file described on page 5. By default, this prop is set to ./http.redirect
check_interval	The HTTPRedirectAgent checks the status of the web servers on the redirect list. If a server doesn’t respond, it will be removed from the redirect list temporarily. It will be added again once the HTTPRedirectAgent pings it and it is responding. By default, this value is set to 300000, which means every 300000 milliseconds the agent will check the redirect destinations on its list.
initial_wait_time	This prop tells the agent how long to wait before it starts checking the redirect destinations. In the case where you are redirecting to yourself, you must wait until the WebServer is fully started before the agent starts pinging it. By default this prop is set to 60000, which means the agent will wait 60000 milliseconds before it starts checking.
service_name	The name this agent uses to register with the ServiceManager. By default, this prop is set to http_redirect.agent

Table 9 - com.neokernel.httpd.HTTPRedirectAgent

com.neokernel.httpd.HTTPPutAgent

When this agent is loaded the server supports traditional HTTP PUT features, saving uploaded files to the directory specified in the html_directory prop. To run the HTTPPutAgent, the WebServer must have a prop called put_agent that matches the service_name that the HTTPPutAgent agent registered with. See the /FileUploadExample folders (in the /demos/Visual Basic Projects and /demos/Visual C# Projects) for an example of generating a file upload form. Not all browsers support this feature so provide an alternate means of file transfer.

service_name

The name this agent uses to register with the ServiceManager. By default, this prop is set to http_put.agent

html_directory

The path to the directory where received files will be stored. By default, this value is set to ./

make_directories

If this prop is set to true, the HTTPPutAgent will create new sub-directories under the server’s html_directory when a client uploads a file to a non-existent directory. By default this prop is set to false

Table 10 - com.neokernel.httpd.HTTPPutAgent

com.neokernel.httpd.LogViewerPage

This agent registers itself with the ServiceManager and generates web reports on the contents of server log files. It analyzes the files in the log_dir directory and uses a mapping file to obtain information about how these files should be displayed. The mapping file maps report names to text search patterns. When a browser accesses this agent, the agent lets the user choose from the reports specified in the map file using a popup menu. When the user submits a request for one of the reports, they are shown all lines in the log files that match the text search pattern for that report. The source code for this agent is included in the /demos directory. This agent will treat every file in the specified directory as a log file.

log_dir	This prop tells the LogViewerPage where to look for the log files.
log_map_file	This is the path to the log.maps file described on page 5. By default, this prop is set to ./log.maps
check_interval	All requests to view the actual lines of log files are processed in real time. This means that the log files are searched when a request for a report is received. However, the “hit counts” displayed on the “choose a report” page are recorded at intervals. This prop defines how often this agent updates these values in milliseconds.
service_name	This is the name that the agent will use as its URL.
is_logger_output	If this prop is set to true, the LogViewerPage will assume that the log files are generated by the Neokernel, resulting in improved formatting of multiple line log entries.
max_show_log_length	This is the maximum length of data (in bytes) that will be shown when a request to view log output is received. The default value is 50000

Table 11 - com.neokernel.httpd.LogViewerPage

com.neokernel.io.FileReporter

This agent captures all output that would normally appear in the console window and writes it to a file. It supports the following Props:

log_file_prefix

The prefix to use when naming log files. Default value is log_

logs_per_file

The maximum number of log entries per file. When logs_per_file has been reached, a new log file is created to hold additional data. Default value is 10000

log_dir

The path where log files will be output to. Default value is ./logs

Table 12 - com.neokernel.io.FileReporter

Note: The com.neokernel.io.ServiceReporter agent is used as the default logger when the Neokernel is running as a windows service. This agent supports the same properties as the FileReporter, but it also logs errors to the Windows Event Log. If the Neokernel is running as a service and you want to make other events appear in the Event Log, use the printEventLogEntry() and debugEventLogEntry() methods of com.neokernel.io.ServiceReporter.

Part Two: Developing Dynamic Web Applications with the Neokernel

Deploying Custom Request Agents

The Neokernel is designed to be extended with custom Agent and RequestAgent classes that you develop. To develop and deploy a custom RequestAgent:

1. Create a class in Visual Basic or C# that extends com.neokernel.ak.RequestAgent and implements the method handleRequest(AgentRequest request).

2. Compile your class and put the new .dll in the same directory as neokernel.exe:

compilinghelloworld_005

Figure 2 - Compiling the HelloWorldAgent.

3. Optionally, add Props for your agent to the neokernel_props.xml file:

a. Create a prop called service_name that will be your agent’s URL, for instance /helloworld.agent.[11]

b. Create a prop called classname set to the name of your class.[12]

4. Start the neokernel.exe application and use your browser to navigate to the agent’s URL as specified in its service_name. The agent’s URL will be in the format:

http://<server’s hostname><agent’s service_name>

Templates for a variety of custom agents and configurations can be found in the /demos folder. The following C# code snippet shows a complete custom agent:

$Text Box: using System; using com.neokernel.nk; public class SimplePage : RequestAgent { public override void initProps() { setDefault("service_name", "/simple.agent"); setDefault("body", "This is output from the SimplePage agent!"); } public override void handleRequest(HTTPAgentRequest request) { String body = getString("body"); request.println( body ); } }$

If a browser requests the URL http://localhost/simple.agent from a server that has started the SimpleAgent from the above example, the browser will display the text

This is output from the SimplePage agent!

Text Box: Imports com.neokernel.nk

Public Class SimpleVBPage
Inherits RequestAgent

Overrides Sub initProps()
setDefault("service_name", "/simple.agent")
setDefault("body", "This is output from the SimplePage agent!")
End Sub

Overrides Sub handleRequest(ByVal Request As AgentRequest)
Dim title, body As String
body = getString("body")
Request.println(body)
End Sub

End Class Here is the same agent implemented using Visual Basic:

Loading New Agent Components

To make a new agent available from your server, put the agent’s assembly in the same folder as the server executable and re-start the server. Agents found in the startup directory are loaded automatically using their default properties unless an agent of the same type has been defined in the neokernel_props.xml file, in which case the properties in the neokernel_props.xml file are used. Agents are typically compiled into .dll or .exe files (called assemblies.) The assembly containing the agent to be loaded must be in the same directory as the neokernel.exe, or a subdirectory of the directory that the neokernel.exe executable was launched from.

How It Works – The Complete Picture

Lets review how everything works together:

1. When a request is received, the Neokernel looks up the ServiceManager to look up the agent that should handle an incoming request.

2. It creates an HTTPRequestObject which it hands-off to the chosen agent by passing it as an argument to that agent’s handleRequest() method.

3. The agent generates a response in the course of it’s handleRequest() method.

Figure 3 – Neokernel task flow including the ServiceManager.

The HTTPAgentRequest passed as an argument when handleRequest() is called serves two purposes: it stores useful data about the incoming request, and it is used to send the response to the client. To write data in response to a request, call the print() or println() methods of HTTPAgentRequest.

The server automatically adds the correct HTTP header and response codes to data sent to the client using the print() and println() methods. A RequestAgent may override this feature if the first text sent in response to a request is HTTP/[13]. If anything besides HTTP/ is printed first, HTTP headers are automatically generated and pre-pended. This ensures that browsers will display the response.

Working with Agent Props

The SimplePage example illustrated how writing Neokernel agents isn't that different from writing other programs in the .NET environment. What is it about agents that make them different? One key difference between agents and other programs is that they maintain an identity defined by agent properties (to review, it is the convention in Neokernel programming that agent properties are called props.)

Before an agent can start using its identity, it initializes its props using the initProps() method. The following code shows initProps() in action:

$Text Box: using System; using com.neokernel.nk; public class SimplePage : RequestAgent { public override void initProps() { setDefault("color", "red"); setDefault("greeting", "Hello, world!"); setDefault("language", "English"); setDefault("service_name", "/simple.agent"); setDefault("body", "This is output from the SimplePage agent!"); } public override void handleRequest(HTTPAgentRequest request) { String body = getString("body"); request.println("I like to speak in " + getString("language")); request.println("My favorite color is " + getString("color")); request.println( body ); } }$

Agents have built-in methods for managing their props. In the above example, setDefault() is called to initialize the color, greeting, and language props of our agent. Instead of the usual message, we can use getString() to retrieve agent props making our message a little more personal. Agents inherit features for Props storage and management from the com.neokernel.props.Props class. Other convenience methods inherited from Props provide built-in ways for agents to set and get individual Props values.

Details on the XML Props File

The neokernel_props.xml file specifies the startup props for agents. Start multiple instances of the same kind of agent by creating multiple props entries with the same classname prop[14]. The agent_id prop controls the order in which agents are started. Agents are started sequentially, so to start a certain agent first, set its agent_id prop to a lower number than other agents in the file. The agent_id prop is optional; if you do not specify an agent_id the server assigns one automatically.

Setting classname in Different Scenarios

When the classname prop isn’t set properly, the server can’t tell what agent you want to load. The two things that affect how to set the classname are the development language used to build the agent (Visual Basic or C#) and the agent’s namespace. Here are examples of the SimpleAgent classname prop in these different scenarios:

If the agent is compiled into the file simpleagent.dll and is not included as part of a namespace, use this syntax:

Visual Basic

simpleagent.SimpleAgent

SimpleAgent

If the agent is compiled into the file simpleagent.dll and is included as part of a namespace, specifically the namespace org.coders, use this syntax:

Visual Basic

simpleagent.org.coders.SimpleAgent

org.coders.SimpleAgent

Getting CGI Parameters

If CGI parameters are included with the incoming request, they can be retrieved using the getString("<parameter>") method of the HTTPAgentRequest using it’s, where <parameter> refers to the name of the CGI parameter that you are getting the value of. The HTTPAgentRequest supports these features because it extends the class com.neokernel.props.Props.[15]

For instance, if a browser requests the URL:

http://localhost/my_agent.agent?param1=value1&param2=value2

…then the HTTPAgentRequest passed to my_agent.agent will include a prop called param1 whose value is the string value1, and a prop called param2 whose value is the string value2. The following code snippet shows how to access the value of param2 in this situation:

Text Box: public override void handleRequest(HTTPAgentRequest request)
{
String value = myAgentReq.getString("param2");
request.println("The value of param2 is " + value);
}

If the CGI parameters don’t have unique names, the parameter values are kept in a single prop as an ArrayList. For example, if the URL

http://localhost/my_agent.agent?param=value1&param=value2

…is requested by a browser, the two values of param are stored in an ArrayList containing the values in the order they were passed. The following code snippet shows how access the CGI parameters when several inputs share the same parameter name:

Text Box: public override void handleRequest(HTTPAgentRequest request)
{
ArrayList paramList;
paramList = (ArrayList) request.getProperty("param");
String value1 = (String)paramList[0];
String value2 = (String)paramList[1];
request.println("The first value of param1 is " + value1);
request.println("The next value for param1 is " + value2);
}

The HTTPAgentRequest also includes support for the following props:

request	The complete request header that was received from the client.
request_id	The request ID.
request_type	The protocol which the request uses (typically HTTP or HTTPS).
path	The path (URL) of the request.
request_server_name	The hostname of the request server.
request_server_port	The port which the server was bound to.
host	The hostname provided with the request that the path is bound to.
request_remote_host	The hostname of the client.
request_remote_addr	The IP address of the client.

Table 13 – Useful props supported by the HTTPAgentRequest.

Tracking Sessions using Cookies and the SessionManager

The HTTPAgentRequest supports setting and managing user cookies. A SessionManager agent must be running for cookies to work properly. The methods in HTTPAgentRequest for handling cookies will only work when a SessionManager is running because the HTTPAgentRequest objects use the SessionManager to keep track of cookies that have been issued[16].

The API documentation for the classes HTTPAgentRequest and SessionManager provide details about each of the cookie management methods and how to use them. The example in the /demos directory illustrates how to set everything up and write code that uses cookies.

Writing Events to the Windows Event Log

The com.neokernel.io.ServiceReporter is used to log information when the Neokernel is running as a service. This reporter provides functions for writing events to the Windows Event Log. When this reporter is being used, any call to error() will appear in the Windows Event Log and will also be written to the text logs stored in the \logs directory. The ServiceReporter's printEventLogEntry() and debugEventLogEntry() methods can be used to output non-error information to the Windows Event Log.

When the Neokernel is running as a service, the following code snippet can be used to print non-error events to the event log:

Text Box: if (NK.DefaultReporter is ServiceReporter)
{
ServiceReporter logger = (ServiceReporter)NK.DefaultReporter;
logger.printEventLogEntry(this, "This has an info icon.");
logger.debugEventLogEntry(this, "This has a warning icon.");
}

Debugging Neokernel Agents in Visual Studio .NET

There are two ways to debug your agents using the Visual Studio .NET debugger. If your application is a stand-alone application, just run the debugger normally and set break points in your agent code. If your agent is compiled as a class library, you can debug it by setting the neokernel.exe executable to be the startup executable in your project settings. Open the Property Pages window for your project; the Visual Basic version of the Property Pages is pictured below.

debugging_image012

Figure 4 – Preparing to debug your Neokernel Agent

Select Configuration Properties, and the Debugging tab underneath it. Set the neokernel.exe to be the startup executable, and set the following as your command line argument:

-startup_dir <path to assembly directory>

…where <path to assembly directory> is the directory where your debug assembly is located (typically the /bin/debug or /bin subdirectory of your project folder.) Don’t forget, modules that will be loaded by the Neokernel must be in the same directory as the neokernel.exe executable or a subdirectory of the directory that contains the executable.

Other Useful Classes

HTTPAgentRequest	This class encapsulates a client request; the server gives an instance of this class to the request agent that will deal with incoming requests from browsers. HTTPAgentRequest provides utility methods for working with streams, headers, cookies, and responses associated with HTTP sessions.
Base64Encoding	This helper class, found in the com.neokernel.util namespace, contains the MIME/64 encoding and decoding methods used by the server when handling authentication. These methods can be used by any agent to encode or decode as needed.
Neokernel	The Neokernel is responsible for managing the life-cycle of all agents, but it does not otherwise keep track of these agents, or attempt to manage any other aspect about them. The Neokernel is an Agent itself, and adheres to standard agent interfaces. It is responsible for loading agent props and creating and starting an agent for each set of props. Each new agent is defined by props, either default or loaded from the configuration file. The startup process is synchronous, ensuring that each startup agent is loaded and started before the next. You can artificially specify loading order by changing an agent’s agent_id prop; agents with a lower agent_id are started first.
ServiceManager	The ServiceManager is an agent that provides a directory service were agents can list themselves as available for use by other agents. The ServiceAgent contains methods used by subclasses to manually register and un-register with the ServiceManager, or automatically handle registration. Many agents, including all agents that work with the WebServer, extend ServiceAgent because of the social needs of agents that depend on other types of agents.
SessionManager	An agent that manages session information and stores session information in Props.
SessionReaper	The SessionReaper is a ScheduledAgent that periodically checks for expired sessions that were created by the SessionManager. Although this agent has no direct interactions with the SessionManager itself, it assumes that session information is stored in the ObjectManager and is tagged with a property called timestamp. Any session object that has a timestamp value (in milliseconds) that is greater than the session_timeout_ms property of the SessionReaper is removed from the ObjectManager. The SessionReaper checks at intervals specified by its schedule prop.
AgentReplicator	An AgentReplicator creates and starts multiple instances of other agents. This is useful when creating a pool of agents for parallel agent architectures.
Scheduler	The Scheduler provides scheduling services to wakeup ScheduledAgents using a customized English-language schedule. The syntax supported by the Scheduler is: date = mm/dd/yyyy time = hh:mm:ss [starting startDateTime] [[every n[y\|m\|w\|d] ] \| [on[[SU][MO][TU][WE][TH][FR][SA]\|[startDay-endDay]]] [[between startTime-endTime every n[h\|m\|s]] \| [at time1,time2,time3]] [until endDateTime] The Scheduler registers itself with the ServiceManager when it starts, and ScheduledAgents automatically schedule themselves when they are started.
FileReporter	The FileReporter saves console output to a file. After this agent has started, no further output appears in the console.
Props	The Props object keeps track of a set of properties – it is similar to a Hashtable but includes many useful methods for storing and returning values as the requested System.Type instead of the generic System.Object type.
FileObjectManager	Manages Props objects and stores them as files, using FilePropsContainers. It is passed a String or FileInfo object that specifies the directory to store Props files in. The files are updated when values in their Props are changed.
XMLPropsList	This class provides features for storing Props and lists of Props in XML files, and loading props from XML files.

Table 14 – Other useful classes included with the Neokernel.

Neokernel Bootstrapping Details

You may run multiple instances of the Neokernel in parallel on the same server. The setup program, accessible from the Add or Remove Programs control panel, will install additional instances of the Neokernel service if the Modify option is selected. If you install an instance of the Neokernel service manually using an admin tool, make sure to use the nkservice.exe executable file and not the neokernel.exe file.

To start the Neokernel from another .NET program, call the main(String args[]) method in the class com.neokernel.nk.NK. By default, the server uses the license file from the application startup directory, and the neokernel_props.xml file in the current working directory. Override this behavior by using the –startup_dir and –startup_file arguments of the main() method.

When the Neokernel starts it does the following:

12. Ensures that a valid Neokernel license file exists in the directory that the application was launched from.

13. Parses the command line arguments.

14. Loads the props list from neokernel_props.xml file if one is found in the executable directory or the specified startup_dir.

15. Probes each .dll and .exe file in the working directory (or the startup_dir if specified) for agents that are not listed in the neokernel_props.xml file.

16. Create default props for any agents it finds that are not listed in the neokernel_props.xml file, and adds them to the list of agents that will be started. If a RequestAgent is listed without a WebServer or ServiceManager, the server will start these supporting agents using the RequestAgent as the index_agent for the created WebServer.

17. After the list has been assembled, the Neokernel starts each agent described in the list of props.

18. You can prevent an agent in the startup directory from starting automatically by creating props for it in the neokernel_props.xml file and setting its startup prop to false.

After the kernel has started, you can manually start and stop agents from your code using the AgentController interface and the methods in NeoKernel class. The bootstrapping examples in the /demos folder show starting the Neokernel from within a VB or C# application and controlling agents after startup.

The information in the documentation is subject to change without notice. If you find any problems in the documentation, please report them to us using the email address support@cometway.com.

This software/documentation contains proprietary information of Comet Way, Inc; it is provided under a license agreement containing restrictions on use and disclosure and is also protected by copyright law. Reverse engineering of the software is prohibited. This software was not developed for use in any nuclear, aviation, mass transit, medical, or other inherently dangerous applications. It is the customer's responsibility to take all appropriate measures to ensure the safe use of such applications if the programs are used for such purposes.

All products and company names referenced in the documentation are used for identification purposes only, and may be trademarks of their respective owners.

Comet Way, Inc.
4800 Liberty Ave
Pittsburgh, PA USA
15224

+1 (412) 682-5282

[1] The properties of Neokernel agents are referred to as props because the object com.neokernel.props.Props is used to represent them in the API and to differentiate them from other properties.

[2] The table shows the default filenames; you can change these filenames by editing the neokernel_props.xml file to change the path where the corresponding agents look for their configuration file.

[3] The Event Viewer is in the System Tools section of the Computer Management console, accessible by right-clicking on My Computer and selecting Manage.

[4] A RequestAgent is any class that extends com.neokernel.nk.RequestAgent.

[5] The WebServer uses a set of default MIME types if a MIME-types configuration file is not present in the startup directory.

[6] Virtual Hosts is not to be confused with multi-homing, which refers to the practice of having multiple upstream internet service providers, and possibly multiple Ethernet cards, installed on a single host. The following is an excerpt from RFC 1122, online at http://www.ietf.org/rfc/rfc1122.txt: “A multihomed host has multiple IP addresses, which we may think of as ‘logical interfaces’. These logical interfaces may be associated with one or more physical interfaces, and these physical interfaces may be connected to the same or different networks.”

[7] The Certificates window is accessible from the tab labeled Content in the Internet Options control panel. The server will use the first certificate it finds in the specified store that is suitable for SSL server authentication.

[8] The localhost-cert.pfx file is an example certificate that is located in the same directory as the server executable (.exe) file.

[9] To start multiple HTTPFileServerAgents, create separate entries in the neokernel_props.xml file for each agent instance that you want to start.

[10] Realms refer to logical areas of a website that require authentication, for instance a directory, specific file, or a set of directories.

[11] Refer to Virtual Hosts on page 6 for more information about setting the service_name prop.

[12] For instance, if you write an agent called SimplePage and compile it into an assembly named WindowsApplication1.dll, the classname prop of your agent in the XML startup file should be WindowsApplication1.SimplePage.

[13] If you are sending custom request headers, the HTTP/ response should be followed on the same line by the HTTP version (1.0 or 1.1) , then the result code (200/304/404/500 etc.), then any other response headers on new lines.

[14] The server will not load an agent if it’s startup prop is false.

[15] The Props class provides a standard interface for working with persistent and transient data, including features for automatically casting between different data types. Other classes in the com.neokernel.props namespace provide for encoding, storage and retrieval of Props in ArrayLists, databases or text files. To work with Props in XML format, use com.neokernel.xml.XMLPropsList.

[16] The built-in SessionManager component is not started by default; if you need cookie and session tracking features edit the neokernel_props.xml file to start the SessionManager starts (by setting the startup prop of the SessionManager to true.)

Neokernel User Guide Release 2.0

Table of Contents

Debugging Neokernel Agents in Visual Studio .NET