Subversion Repositories openrisc

This package provides an extensible, small footprint, full featured HTTP

server for eCos. Many of these features can be

disabled via the configuration tool, thus reducing the footprint of the server.

The server has been written for the FreeBSD network stack.

This ATHTTP implementation provides the following features:

  GET, POST and HEAD Methods

  File system Access

  Callbacks to C functions

  MIME type support

  CGI mechanism through the OBJLOADER package or through a

                  simple tcl interpreter

  Basic and Digest (MD5) Authentication

  Directory Listing

  Extendable Internal Resources

Ecos tables are used extensively throught the server to provide a high degree

of customization.

In order to start the web server, the user needs to call the function:

cyg_httpd_start();

in the application code. The server initialization code spawns a new

thread which calls init_all_network_interfaces() to

initialize the TCP/IP stack and then starts the deamon. The function is safe

to call multiple times.

The server has an internal table with all the recognized mime types. Each time

a file or an internal resource is sent out by the server, its extension is

searched in this table and if a match is found, the associated MIME type is

then sent out in the header.

The server already provides entries for the following standard file extensions:

'html', 'htm', 'gif', 'jpg', 'css', 'js', 'png'

and the user is responsible for adding any further entry. The syntax for

adding an entry is the following:

CYG_HTTPD_MIME_TABLE_ENTRY(entry_label, extension_string, mime_tipe_sting);

entry table      : an identifier unique to this entry

extension string : a string containing the extension for this entry

type_string      : the mime string. The strings for many more mime types

                   is included in a file in the "doc" directory.

The following is an example of how to add the Adobe Portable Document Format

pdf MIME type to the table:

CYG_HTTPD_MIME_TABLE_ENTRY(hal_pdf_entry, "pdf", "application/pdf");

For chunked frames, which are generally used inside c language callbacks, there

is no file name to match an extension to, and thus the extension to be used

must be passed in the cyg_httpd_start_chunked() call. The

server will then scan the MIME table to find a MIME type to match the extension.

For example, to start a chunked transfer of an html file,

the following call is used:

cyg_httpd_start_chunked("html");

In any event, it is the responsibility of the user to make sure that a match to

all used extensions is found in the table search. Failing this,

the default MIME type specified in the CYGDAT_NET_ATHTTPD_DEFAULT_MIME_TYPE

string is returned.

The server allows the association of particular URLs to C language callback

functions. eCos tables are used to define the association between a URL and its

corresponding callback. The syntax of the macro to add callback entries to

the table is:

CYG_HTTPD_HANDLER_TABLE_ENTRY(entry_label, url_string, callback);

entry table      : an identifier unique to this entry.

url_string       : a string with the extension url that will be appended to the

                   default directory.

callback         : a function with a prototype:

                   cyg_int32 callback_function(CYG_HTTPD_STATE*);

                   Return value is ignored - just return 0.

CYG_HTTPD_STATE* is a pointer to a structure that

contains, among others, a buffer (outbuffer) that can be used to send data

out. The definitions of the structure is in http.h.

The following is an example of how to add a callback to a function myForm()

whenever the URL /myform.cgi is requested:

CYG_HTTPD_HANDLER_TABLE_ENTRY(hal_cb_entry, "/myform.cgi", myForm);

and somewhere in the source tree there is a function:

cyg_int32 myForm(CYG_HTTPD_STATE* p)

{

   cyg_httpd_start_chunked("html");

   strcpy(p->outbuffer, "eCos Web Server");

   cyg_httpd_write_chunked(p->outbuffer, strlen(p->outbuffer))

   cyg_httpd_end_chunked();

}

This function also shows the correct method of using the chunked frames

API inside a c language callback and also shows the use of outbuffer to

collect data to send out.

Chunked frames are useful when the size of the frame is not known upfront.

In this case it possible to send a response in chunks of various sizes, and

terminate it with a null chunk (See RFC 2616 for details). To use chunked

frames, the cyg_httpd_start_chunked() function is used.

The prototype is the following:

ssize_t cyg_httpd_start_chunked(char *);

The only parameter is the extension to use in the

search for the MIME type. For most files this will be "html" or "htm" and

it will be searched in the MIME table for an approriate MIME type that will

be sent along in the header. The function returns the number of bytes sent

out.

The chunked frame must be terminated by a call to

cyg_httpd_end_chunked():

void cyg_httpd_end_chunked()(void);

In between these two calls, the user can call the function

cyg_httpd_write_chunked() to send out data any number of

times. It is important that cyg_httpd_write_chunked() be

the only function used to send data out for chunked frames. This

guarantees that proper formatting of the response is respected.

The prototype for the function is:

ssize_t cyg_httpd_write_chunked(char* p, int len);

The 'char*' points to the data to send out, the 'int' is the length of the

data to send.

In the case in which the size of the data is known upfront, the

callback can instead create the header with a call to

cyg_httpd_create_std_header() with the following

prototype:

void cyg_httpd_create_std_header(char *ext, int len);

extension   : the extension used in the search of the MIME type

len         : length of the data to send out

and use

cyg_httpd_write() to send data out to the client. The

 prototype of cyg_httpd_write() is the same as

cyg_httpd_write_chunked()

The web server allows writing of pseudo-CGI programs. This is helpful in order

to modify the functionality of the server without having to recompile it and

reflash it.

One way to implement CGI is, of course, the C language callback mechanism

described above: This assumes, of course, that all the callbacks are written

by compile time and cannot be modified later on. Another way to perform the

same functionality is the use of a library in the form of an object file.

These object files reside in the file system and are loaded, executed and

unloaded on demand.

Yet a third way is the use of a scripting language. Since full fledged

implementation of the most popular scripting languages such as Python or Perl

are too large for most embedded systems, a slim down implementation of tcl

was chosen for this server. Most of the tcl functionality is still there,

and makes writing cgi a lot easier.

In order to limit the footprint of the operating system support for both

the objloader and the tcl script for dealing with cgi files can be

independently selected out. Tcl support in particular increases the memory

requirements considerably.

In order to use the cgi mechanism the CYGPKG_OBJLOADER must be included

when building the operating system. This will enable the proper option in the

configuration tool and if selected, the necessary code will be compiled

in the eCos kernel. The user will then have to compile the necessary libraries

and place them in the file system under a directory defined by

CYGDAT_NET_ATHTTPD_SERVEROPT_CGIDIR.

When a request is made, the web server checks if the root directory of the

requested URL is inside the CYGDAT_NET_ATHTTPD_SERVEROPT_CGIDIR directory.

If so, the server assumes that the user requested a cgi file and looks into the

directory to see if a library by the same name is present, and if so load it

and tries to execute a function inside the library with the following prototype:

void exec_cgi(CYG_HTTPD_STATE *)

The pointer CYG_HTTPD_STATE* gives access to the socket

data: The user will use this pointer to access the 'outbuffer' and use it to

copy data to send data out.

When using the OBJLOADER package within the HTTP server a number of functions

are automatically added to the externals table of the OBJLOADER package. These

functions are likely to be used inside the library and the relocator need to

have a pointer to them. In order to add more functions, see the OBJLOADER

documentation. The complete list of the functions automatically added is:

  cyg_httpd_start_chunked()

  cyg_httpd_write_chunked()

  cyg_httpd_end_chunked()

  cyg_httpd_write()

  cyg_httpd_find_form_variable()

  cyg_httpd_find_ires()

  cyg_httpd_send_ires()

  diag_printf()

  cyg_httpd_format_header()

  cyg_httpd_find_mime_string()

Every time the web client issues a GET or POST request for a file with an

extension of '.o'in the /cgi-bin directory (or whatever path the user chooses

to hold the libraries) then the library by that name is loaded, run and

when the execution is over, it is dumped from memory.

The library must be compiled separately, using the same toolchain used to

compile the server and then added to the file system.

In order to reduce the footprint of the server, CGI through OBJLOADER

can be compiled out by unchecking CYGOPT_NET_ATHTTPD_USE_CGIBIN_OBJLOADER

in the configuration tool.

A small tcl interpreter has been added to the web server, and it can

be used to write simple cgi scripts. The interpreter is admittedly very

minimal, and it is only useful for very simple applications, but it is an

excellent starting point for further development.

In order for the scripting language to be useful, it has to access

the form variables passed on during the GET or POST request. Because of

this, all form variables registered with the CYG_HTTPD_FVAR_TABLE_ENTRY()

macro are accessible via tcl. For example, if we have registered a

form variable called foo, and during the GET request we are defining foo

as being "1":

GET /myForm.cgi?foo=1

then tcl will be able to access the variable foo as $foo. The data

in the body of a POST request is also accessible through the use of the variable

$post_data. This is useful if the data is not in "multipart/form-data"

and tcl has to perform any type of processing on the data itself.

In order to send back a response to the client a few functions have been

added to the interpreter. These functions are:

start_chunked "extension";

"extension" is a string used to search the

table of the mime types. For example, to send back to the client an HTML file,

we can use: start_chunked "html";

write_chunked content;

content is a string to send back to the client.

end_chunked;

No parameters. Send back an end of frame to the client.

The following example demonstrates how to send a log file in the file

/ram/log to a web client. It replaces

newline characters with <br> so that it is formatted on the

browser correctly.

start_chunked "html";

set fp [aio.open "/ram/log" r];

$fp seek 0 end;

set fsize [$fp tell];

$fp seek 0 start;

set data "abcxxx";

set data [$fp read $fsize];

$fp close;

set data [string map {\n <br>} $data];

set datax "";

append datax "<html><body>" $data "</body></html>";

write_chunked $datax;

end_chunked;

The above file should exist on a filesystem

on the embedded target within its cgi-bin

directory, for example as /cgi-bin/hello.tcl. Thereafter

it may be accessed at the URL

http://TARGET_NAME/cgi-bin/hello.tcl.

The server supports both Basic (base64) and Digest (MD5) authentication,

although they have not been tested with all clients. In this implementation,

the contents of certain directories of the file system can be protected, such

that the user will be required to issue a username/password to access the

content of the directory.

To protect a directory with a basic authentication, there is a

specific macro:

CYG_HTTPD_AUTH_TABLE_ENTRY(entry, path, domain, un, pw, mode)

entry            : an identifier unique to this entry.

path             : the path to the directory whose content must be

                    authenticated before it is sent out

domain           : a domain identifier for this directory.

un               : username for authentication

pw               : password for authentication

mode             : CYG_HTTPD_AUTH_BASIC for base64 encoding or

                   CYG_HTTPD_AUTH_DIGEST for MD5 encoding

for example, to require basic authentication of the content of directory

"/ecos/" with a username of "foo" and password "bar", the following is used:

CYG_HTTPD_AUTH_TABLE_ENTRY(hal_domain1_entry,          \

                           "/ecos/",    "ecos_domain", \

                           "foo",       "bar",         \

                           CYG_HTTPD_AUTH_BASIC);

Any request for a file in the directory /ecos/ will now trigger a

credential check. These credentials, once provided, are automatically sent by

the client for every request within the particular domain.

It must be noticed that the path name set in the macro is relative to the

HTML document directory, CYGDAT_NET_HTTPD_SERVEROPT_HTMLDIR and it is the

first part of the path provided by the client request (including the leading

slash).

In order to reduce the footprint of the server, authentication

is not enabled by default, and so the option CYGOPT_NET_ATHTTPD_USE_AUTH must

be used to enable support for basic and digest authentication.

The MD5 digest authentication support is implemented using the RSA

Data Security, Inc. MD5 Message-Digest Algorithm. Derivative works with

MD5 digest authentication included must be identified as "derived from the

RSA Data Security, Inc. MD5 Message-Digest Algorithm" in all material

mentioning or referencing the derived work. See the file md5.c within this

package for license details.

If the user issues a "GET" request with a URL terminating in a slash, the

server will try to locate one of the following index files in the directory,

choosing one in the following order:

  index.html

  index.htm

  default.html

  home.html

If any of these files is found, its contents are sent back

to the client. If no such file is found the server uses the user-provided

index file name (if any is specified with the CYGDAT_NET_ATHTTPD_ALTERNATE_HOME

setting. Failing all this a directory listing is sent.

Trailing slash redirection for directory names is supported.

In order to reduce the footprint of the server, directory listing can

be disabled by unchecking CYGOPT_NET_ATHTTPD_USE_DIRLIST. The savings are

substantial since directory listing also makes use of a few internal

resources (gif files) which are also compiled out.

The server will automatically try to parse form variables when a form is

submitted in the following cases:

  In a GET request, when the URL is followed by a question

                  mark sign

  In a POST request, when the the 'Content-Type' header line

                  is set to 'application/x-www-form-urlencoded'

The variable names to look for during the parsing are held in

an eCos table. In order to take advantage of this feature, the user first

adds the variable names to the table, which also requires providing a buffer

where the parsed value will eventually be stored. The values will then be

available in the buffers during the processing of the request, presumably in

the body of a c language callback or CGI script.

For example, if the user wants two form variables, "foo" and "bar", to

be parsed automatically, those variable names must be added to the table

with the following macro:

CYG_HTTPD_FVAR_TABLE_ENTRY(entry, name, buffp, bufflen)

entry            : an identifier unique to this entry.

name             : name of the form variable

buffp            : a pointer to a buffer of characters where to store the value

                   of the form variable.

bufflen          : The length of the buffer. Must include a trailing string

                   terminator.

or, in the specific instance mentioned above:

#define HTML_VAR_LEN   20

char var_foo[HTML_VAR_LEN];

char var_bar[HTML_VAR_LEN];

CYG_HTTPD_FVAR_TABLE_ENTRY(hal_form_entry_foo, "foo", var_foo, HTML_VAR_LEN);

CYG_HTTPD_FVAR_TABLE_ENTRY(hal_form_entry_bar, "bar", var_bar, HTML_VAR_LEN);

and after the GET or POST submissions, the list will contain the value

for "foo" and "bar" (if they were found in the form data.) It is the

responsability of the user to make sure that the buffer is large enough

to hold all the data parsed (including the string terminator). The parser will

write only up to the length of the buffer minus one (the last being the

terminator) and discard any additional data.

The values parsed are likely going to be used in c language callback, or

in CGI files. In a c language callback the user can directly access the pointers

of individual variables for further processing, keeping in mind that the parsing

always result in a string of characters to be produced, and any conversion

(e.g. from strings to integer) must be performed within the callback. In

a TCL script the user can just access a variable by its name. For example,

in the case of the variables 'foo' and 'bar' shown above, it is possible

to do something like 'write_chunked "You wrote $foo". The data that was sent in

the body of a POST request is accessible in through a variable called

'post_data'. In CGI functions

implemented using the objloader the pointers to the

variables cannot be accessed directly, since the library will likely not

know their location in memory. The proper way to access them is by using the

cyg_httpd_find_form_variable() function from within the library:

char* cyg_httpd_find_form_variable(char* name)

name             : name of the form variable to look up

returns a pointer to the buffer, or 0 if the variable was not found.

When using the OBJLOADER package within the web server, an entry

for the cyg_httpd_find_form_variable() function is automatically added to the

externals table the OBJLOADER for relocation. See the OBLOADER paragraph of

the ATHTTP user's guide for the full list of the exported functions.

In order to avoid stale data, all the buffers in the table are cleared

before running the parser and thus any variable in the list that was not

assigned a new value dureing the request will be an empty string.

When the server does not use a file system the user must be responsible

to provide a C language callback function for each URL that will be

requested by the client. This means locating the data and sending it out

using either cyg_httpd_write() or

cyg_httpd_write_chunked().

In order to simplify this process the server allows registering

any number of URLs as internal resources, by providing the URL name, the

pointer to the resource data and its size. When a URL is requested the

server will look it up among all internal resources, and if found, it

will send out the resource.

Internal resource can also be used along with a file system. In this

case the file system is searched first, and if a file is found, it it

sent. If a file is not found, the internal resources are searched and

if a match if found it is sent.

The drawback of this approach is, of course, that all these

resources are going to add to the size of the operating system image, and thus

it should be used only when memory is not a major constraint of the

design.

As always, to provide this type of customization, ecos tables are used.

The format for adding a new resource to the internal table is the following:

CYG_HTTPD_IRES_TABLE_ENTRY(entry, name, buffp, len)

entry            : an identifier unique to this entry.

name             : name of the URL including leading '/'

buffp            : a pointer to a buffer of characters where to store the value

                   of the form variable.

len              : size of the array

As an example, if the user wants to provide his own web page by

hardcoding it in the application code, here is how he would do it:

#define MY_OWN_HOME_PAGE "eCos RTOS"

CYG_HTTPD_IRES_TABLE_ENTRY(cyg_httpd_ires_home,       \

                           "/index.html",             \

                           MY_OWN_HOME_PAGE,          \

                           9);

The extension of the file name determines the MIME type to be used for

internal resources.

When using directory listing you are implicitly making use of internal

resources. The small icons that appear to the left of file names and

directories are internal resources. Unchecking CYGOPT_NET_HTTP_USE_DIRLIST

will prevent the addition of these files.

In order to use internal resources, a generic file must first be

turned into a c language array, which is then compiled in the application

code. To create this array you can use the tcl script that comes with the

ecos distribution at packages/fs/rom/current/support/file2.tcl.