Feedback

Timeline

It's not the size of the dog in the fight, it's the size of the fight in the dog. Mark Twain

To Upgrade G-WAN: (a) copy the ./include files and the new gwan executable from the archive and then (b) run G-WAN once without the -r switch to make sure that all your servlets and handlers compile without modifications.


Updated detection of the number of CPUs and CPU Cores

Feb
05
2012

Changes in G-WAN v3.2.5/Linux: Candidate for Production

  • fixed the detection for recent Intel CPUs (the problem was with CPUs, not CPU Cores).

Please let me know what you get with AMD CPUs (I don't have any so I can't test them). You can see what G-WAN has detected in the gwan.log file:

 [Sun Feb 05 14:51:39 2012 GMT] 1 Intel(R) Xeon(R) CPU W3680 @ 3.33GHz (6 Cores/CPU, 2 threads/Core)
 [Sun Feb 05 14:51:39 2012 GMT] Using 6 worker threads

G-WAN's by-default policy is to use one worker thread by CPU Core.

Thank you Thomas for the heads-up.


Ubuntu 11 (beta), get_env() values for G-WAN scripts threading

Feb
03
2012

Changes in G-WAN v3.2.3/Linux: Candidate for Production

  • added support for Ubuntu 11 (this Ubuntu version is still in beta, the long-term release is 10.04)
  • added the following get_env() values for G-WAN scripts:
  •  NBR_CPUS,    // int NBR_CPUS;    // total of available CPUs
     NBR_CORES,   // int NBR_CORES;   // total of available CPU Cores
     NBR_WORKERS, // int NBR_WORKERS; // total of server workers
     CUR_WORKER,  // int CUR_WORKER;  // current worker thread: 1,2,3...
    

    NBR_CORES = (number of Cores per CPU) * (number of CPUs).

    NBR_WORKERS = number of worker threads created (by default this is NBR_CORES, but it can be changed by renaming the gwan executable to gwan_2 for 2 worker threads, hence the NBR_WORKERS value which lets G-WAN scripts know it).

    CUR_WORKERS lets scripts get the number of the current worker thread. This is useful to make thread-safe code.

Thank you Philippe and Thomas.


Very large files download fix

Feb
02
2012

Changes in G-WAN v3.2.2/Linux: Candidate for Production

  • fixed the download of very large files.

Thank you Carlos for the tests!


Minor bug fixes, G-WAN scripts speedup, smaller memory footprint

Feb
01
2012

Changes in G-WAN v3.2.1/Linux: Candidate for Production

  • added the VHOST_ROOT definition to gwan.h and made sure that it works as expected
  • tested various strategies to achieve better performance in the G-WAN server and its scripts
  • added remarks in the hello.d example and the related FAQ for the 'D' programming language
  • reduced the pollution of the scripts namespace for better performance and memory footprint.

Thanks to Thomas and Ersun!

Philippe, a long time BASIC fan, has suggested that G-WAN should let us all use this venerable programming language (30 years ago I wrote "STYLED" a GUI coded in asm for the BASIC language). His efforts and the help of people who worked hard at reviving the joys of BASIC will make his dream come true!

This BASIC being compiled (and used from G-WAN), it will smoke many 'modern' Web development languages.


Speedup, VHOST_ROOT, FONT_ROOT, and the 'D' programming language

Jan
31
2012

Changes in G-WAN v3.1.31/Linux: Candidate for Production

  • Re-organized portions of the 64-bit code to get the same performance than in the 32-bit release
  • added a VHOST_ROOT value for get_env() for even quicker access to this G-WAN-relative path
  • this new release fixes a recent glitch which made the angel process (-d) fail to restart the child
  • added (experimental*) support for the 'D' programming language (see the hello.d example)
  • made the path returned by get_env(argv, FONT_ROOT) be absolute instead of relative.

Thanks to Thomas and Ersun!

[*] The D programming language requires you to write "extern (C)" before all (translated in D) gwan.h prototypes (see the hello.d example). The D import replacement for the C include directive could probably be used – if only its file-system mapping was documented). Any help in this matter is appreciated.


libcurl, flv.c, Async. [client] calls, warning, Obj-C, get_env(), etc.

Jan
27
2012

Changes in G-WAN v3.1.27/Linux: Candidate for Production

  • added a new servlet example /csp/curl.c for libcurl (which supports many protocols)
  • fixed an flv.c handler crash when a large video was heavily seeked in a random manner
  • removed a compiler flag which prevented the async. BSD socket [client] calls from working
  • made it possible to disable a listener by just adding an underscore ("_") at its beginning
  • added an explicit warning for those directories locked by file permissions, SELinux, etc.
  • added a new entry in the server report() function to report the name of the Unix OS
  • added a new procedure for Obj-C/C++ on Linux distributions which use different paths
  • the get_env() function has been simplied and all the servlets/handlers updated:
  • 1
    2
    3
    4
    5
    // that was for prior versions
    u32 *states = (u32*)get_env(argv, US_HANDLER_STATES, 0);
    
    // note the missing parameter at the end
    u32 *states = (u32*)get_env(argv, US_HANDLER_STATES);
    

    Far too many users were confused by the two ways to use pointers with this function.

Thanks to Carlos, Eric, Fifi, Tim and Bob!

The real async. BSD socket [client] calls magic has been recoded in asm and will be released once fully tested.

The D, COBOL and Scheme programming languages are on their way to be used from G-WAN!


gwan.log first timestamp entry typo, Listener's Handler initialization

Jan
24
2012

Changes in G-WAN v3.1.24/Linux: Candidate for Production

  • fixed the initialization of the init() states with several Listeners (only the first listener was initialized)
  • fixed a rounding error which sometimes added garbage at the end of Gzip-compressed HTML pages
  • fixed the first gwan.log timestamp typo introduced in v3.1.23, making it difficult to open/read the file.

Thanks to Philippe and Volodymyr for the tips!


Handlers freeing at G-WAN shutdown, sendemail() typo fixed

Jan
23
2012

Changes in G-WAN v3.1.23/Linux: Candidate for Production

  • fixed G-WAN Connection Handlers that were not properly freed at program shutdown
  • fixed a typo introduced in v3.1.22 and which was breaking the sendemail() command.

Thanks Tim, Steven, Rick and Thomas!


Better error for empty script file, or script without main()

Jan
22
2012

Changes in G-WAN v3.1.22/Linux: Candidate for Production

  • now G-WAN reports why an empty script or a script without main() failed to run
  • added whether the G-WAN process is 32-bit or 64-bit in the server_report() function
  • added prettier warning / error formatting for the compilation / linking phases
  • since I was asked the question: the best terminal settings to use are:
    • Built-in Scheme: "White on Black"
    • Text color / Bold Color: Same as text color
    • Palette: "Linux Console"
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    Error: test.c
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    /opt/gwan/0.0.0.0_8080/#0.0.0.0/csp/test.c: In function 'main':
    /csp/test.c:16: error: 'i' undeclared (first use in this function)
    
    12| int main(int argc, char *argv[])
    13| {
    14|    xbuf_cat(get_reply(argv), "Hello World");
    15|   
    16!    return i;
    17| }
    

Thanks to Ersun, Eli, Thomas and Tim!


G-WAN Connection Handlers can now use C++, Objective-C/C++

Jan
21
2012

Changes in G-WAN v3.1.21/Linux: Candidate for Production

  • G-WAN Connection Handlers can now use C++, Objective-C/C++ (not only C).

Thanks to Thomas for the suggestion!

Note that G-WAN Content Handlers (like flv.c) still must use C (extending them will be done later).


Static requests' Query string for Handlers

Jan
20
2012

Changes in G-WAN v3.1.20/Linux: Candidate for Production

  • allowed handlers to use get_env(QUERY_STRING) for static requests like:
  •   http://trustleap.com:80/en_timeline.html?123#gwan_api
      ----   ------------- -- ---------------- --- --------
        |           |       |         |         |      |
      protocol   hostname  port      path     query  fragment (section within document)
                   /        /         |         |          \
         SERVER_NAME  SERVER_PORT  REQUEST  QUERY_STRING   FRAGMENT_ID
    

Thanks Arek!


Compilation Warnings, 32-bit crash reports

Jan
19
2012

Changes in G-WAN v3.1.19/Linux: Candidate for Production

  • allowed G-WAN 32-bit to issue full crash reports (I am now using G-WAN 64-bit)
  • filtered system replies containing UNICODE characters (what a stupid idea, really)
  • fixed a typo preventing any setup $TMPDIR substitute from being used by G-WAN
  • fixed a JSON glitch which prevented large data sets parsing/rendering to work
  • allowed compilation warnings even when no compilation error takes place.

Thanks to Paul and David for the heads-up!


HTML minifying, MIME types, JSON rendering, Linux translations, "%v" trick

Jan
18
2012

Changes in G-WAN v3.1.18/Linux: Candidate for Production

  • fixed a JSON printing glitch ("10.0" was stored as "10." and is now stored as "10")
  • fixed HTML minifying eating spaces because of an undetected HTML "</script>" tag
  • fixed "./gwan -r argv.c a b c" to let the "-r" switch use argv[] command-line arguments
  • added get_env()'s REQUEST_TMO and MIN_SEND_SPEED to tune the DoS shield
  • added support for $TMPDIR on those systems where the /tmp directory may be locked
  • added new MIME types (*.m, *.mm, etc.) to cope with the newly supported languages
  • added localization checks to make all the system replies make sense for G-WAN
  • added a filter to remove repetitions in s_snprintf()/xbuf_xcat()'s "%v" outputs:
  •      0: 00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00 | ................
        16: *              (we skip 250 lines here)
     4,032: 00 00 00                                         | ...
    

    This is useful to stip-down KBs-long junk requests before logging. "%v" lets G-WAN keep relevant data (binary contents or obvious junk requests) in a readable form – without the bloat.

Thanks to Aris, Ersun, Thom, Andy, Ferhat, Emmanuel and Mike!


Minor enhancements and 32-bit bug fix

Jan
09
2012

Changes in G-WAN v3.1.9/Linux: Candidate for Production

  • allowed CSS and Javascript files to use extended character sets (UTF-8)
  • gzip compression will now work in all the cases as long as files are < 2 MB.

Thanks to Michael and Manuel for their feedback!


Minor enhancements and 32-bit bug fix

Jan
08
2012

Changes in G-WAN v3.1.8/Linux: Candidate for Production

  • fixed a 32-bit issue with v3.1.6/7 which prevented the compilation of scripts
  • added a test to prevent two scripts from conflicting while using the same name
  • added more checks to allow G-WAN to run with artistic folder and file names.

Thanks to Stefan, Arek and Slavomir for their tests!


Much faster compilation, faster log files, AMD CPUs boost, bug fixes

Jan
07
2012

Changes in G-WAN v3.1.7/Linux: Candidate for Production

  • fixed a glitch with virtualization layers which require full path to work
  • fixed a glitch with virtualization layers which rename the CPU ID string
  • added a missing test to let gwan run without access/error log files enabled
  • allowed the compilation of scripts which have exotic filenames (like spaces).

Thanks to Ersun, Slavomir, Stefan and DeepInit (Arek) to point those issues!


Much faster compilation, faster log files, AMD CPUs boost, bug fixes

Jan
06
2012

Changes in G-WAN v3.1.6/Linux: Candidate for Production

  • optimized the compilation of scripts (whatever the language) by a factor three
  • optimized the log file rotation security (see previous release) for less overhead
  • removed an Intel-specific test which disabled an optimization for AMD processors
  • fixed the daemon mode which did not start due to a workaround needed for 64-bit
  • fixed bmp_t accidentally not updated in gwan.h (it was correctly documented here)
  • fixed an xbuf_insert() 'universal' interface typo which was breaking the feature
  • replaced the C/C++-like Objective-C/C++ examples by Thomas Meitz's code
  • added a G-WAN Objective-C + GnuStep example written by Thomas Meitz.

Thanks to Eric, Lucas and Thomas for the feedback!

A maintenance downtime is scheduled for today's afternoon (~4-6pm) to upgrade the version of the gwan.ch Linux distribution (no longer supported by its vendor). Since G-WAN v3 requires glibc 2.11+, supported by this new Linux version, the migration will also let gwan.ch use G-WAN v3.


Better compilation warning/error reporting, bug fixes

Jan
03
2012

Changes in G-WAN v3.1.3/Linux: Candidate for Production

  • added cleaner warning and error descriptions during compilation of C scripts
  • fixed the 'graceful kill' (sudo ./gwan -k) which crashed with the 64-bit release
  • fixed disabled gzip compression depending on internal state and file size
  • fixed a narrow case where log files would not be rotated at gwan's startup
  • tried to further secure log files rotation at 0:00am (feedback is welcome).

In chronologic order, thanks to Eric, Tim, Alex, Michael and Tomas for their useful reports!


64-bit, C++, Objective-C/C++, scripts optimizations, bug fixes

Jan
01
2012

Changes in G-WAN v3.1.1/Linux: Development Release

As G-WAN now supports other programming languages than C the URI format to use is no longer "/csp/hello" but rather "/csp/hello.c" or "/csp/hello.cpp" (just like with other application servers which use suffixes like jsp, php, py, or rb). Clients can therefore query "/csp/hello.c" or "/csp/hello.cpp" – and get different results by running different programs.

  • added a 64-bit release (no memory limit, no more system or third-party library issues, etc.)
  • added much more detailled warning and error descriptions during compilation of C scripts
  • added multi-pass optimization and native CPU intrinsics for the local CPU type in C scripts
  • added a 'universal' interface for G-WAN scripts to support more compiled languages soon:
    • Asm ............... *.asm
    • Ada ............... *.ada
    • BASIC ............. *.bas
    • C ................. *.c
    • C++ ............... *.cpp
    • COBOL ............. *.cob
    • D ................. *.d
    • Fortran ........... *.for
    • Google Go ......... *.go
    • Java .............. *.java
    • Modula ............ *.mod
    • Objective-C ....... *.m
    • Objective-C++ ..... *.mm
    • Pascal ............ *.pas
    • Scheme ............ *.scm

    Today, G-WAN natively supports Assembly, ANSI C, C++ and Objective-C/C++ while Javascript, Lua, Google Go, and Python are supported by G-WAN handlers which call a library that implements the language (click the links for each handler). Native support is more convenient to use and executes faster, hence the native implementation of languages already supported by handlers.

    PHP will gladly be added to the list above the day a thread-safe PHP-embed library (or even better, an up-to-date PHP compiler) is made publicly available.

  • fixed the first first line of comment minified in JS files (minification only works in "sudo ./gwan -d" mode)
  • fixed a bug in the floodfill algorithm used for pie and ring charts (the charts still need a bit of love)
  • fixed a bug happening when Gzip is supported but the payload content-type can't be compressed
  • fixed a bug happening during a maintenance process aimed at defragmenting the application memory
  • fixed the close() BSD async. call that blocked further connections in client mode after a first call.

More 'Hello World' examples using the new 'universal' interface to natively use new programming languages will come soon (see hello.c, hello.cpp, hello.m and hello.mm).

All the old examples have been rewritten to compile without errors and warnings. You will probably have to work a bit to polish your code too – that's all the value of extended compiler diagnostics.

If you were using Linux 32-bit to get better G-WAN crash reports, it is advised to switch to Linux 64-bit (much faster than Linux 32-bit) as the G-WAN 64-bit release is now on par with the 32-bit release in this matter.


get_env(FRAGMENT_ID), Accept-Encoding (Firefox), extended 'trace'

Oct
15
2011

Changes in G-WAN v2.10.15/Linux: Development Release

  • fixed the "Accept-Encoding:" HTTP header which failed on Firefox and worked on Chrome and others
  • added parsing of the SDCH encoding type, see served_from.c (G-WAN does not implement the encoding)
  • added the '-t' command-line option to log all client requests in the 'trace' file (when access.log is not enough)
  • added parsing of URL fragment (the last '#') that can now be queried with get_env(FRAGMENT_ID).

HTTP fragments use the final '#' character. Note that for /csp? requests a fragment will be part of the last query parameter (and not available with a get_env(FRAGMENT_ID) call).

The new -t command-line switch has an enormous educational value. Some clients are not short of imagination. Just in case, you will have a time stamp, their IP address and the whole request.


csp script improperly updated when invoked for first time

Oct
13
2011

Changes in G-WAN v2.10.13/Linux: Candidate for Production

  • fixed a pointless (timeout calculation error) update of all C scripts when called the first time
  • enhanced the adaptative Lorenz-Waterwheel for even more speed and less CPU resources consumption.

Cache miss fix

Oct
12
2011

Changes in G-WAN v2.10.12/Linux: Development Release

  • fixed the "Accept-Encoding: gzip" parsing (gzip was sometimes ignored, thanks to Aris and Eric)
  • fixed a v2.10.11 timer issue making cached entries expire at an earlier time than the expected time.

Pipelined/URI requests sanity checks, better cache management

Oct
11
2011

Changes in G-WAN v2.10.11/Linux: Development Release

  • added memory checks to limit memory consumption of huge loads of huge data sets (thanks Jacques)
  • Fredrik's "infinite loop" was merely HTTP pipelining patiently consuming looong "/csp/aaa..." junk (no longer)
  • fixed Fredrik's "/.." unsuccessful directory traversal attempt (stopped at level 1) – the test was wrong by 1 character
  • from the same report, escaping "/%31+%32=%33 HTTP/1.0" was checked but seems to work as expected.

Fast startup with huge /www folders, JSON empty array bug fix

Oct
10
2011

Changes in G-WAN v2.10.10/Linux: Development Release

  • G-WAN no longer conducts /www analysis at startup, leading to instant availability with huge data sets
  • fixed a JSON renderer bug (missing '[' with empty arrays), thank you Griffin for the relevant report.

New serve cache.c servlet example, graceful servlet crash reports

Oct
8
2011

Changes in G-WAN v2.10.8/Linux: Development Release

  • G-WAN v2.10.7 disabled graceful servlet crash reports (for debugging) – this is enabled again in v2.10.8
  • added a cache.c servlet example to show how to serve existing cache entries without data copy from servlets

Update: on Oct. 10th, Fredrik Widlund's blog published inaccurate information that the text below attempts to correct (Fredrik rejected all suggestions):

 Here are the most serious claims, made for v2.10.7:

 "A) A buffer overflow issue exists in the routine handling URL encoding for the "csp" 
     (so called G-WAN servlets) sub-directory."

 "B) SIGPIPE signals were not handled correctly. Exploiting the vulnerability resulted 
     in denial of service."

 What is going on here – and why v2.10.8 fixed it by merely re-enabling signal handlers?
 Even more oddly, why previous versions of G-WAN were not affected?

 When G-WAN receives a dynamic request, before running user-defined code (C servlets),
 it installs a signal handler to catch faults in order to produce "graceful crash reports" 
 instead of having the crash stop the server.

 Without this handler the server crashes because critical structures are initialized there.

 And this is precisely what Fredrik was doing: triggering a segfault that was not handled 
 by v2.10.7 (which had a lifespan of less than 24 hours) but which is properly handled 
 by v2.10.8+ – and by all the versions that preceded 2.10.6/7.

 So, let's expose the value of Fredrik's "research":
 ----------------------------------------------------
 "Vulnerability" A: 
 ==================
 v2.10.7 crashes because it lacks its (accidentally disabled) signal handler to handle 
 crashes gracefully instead of stopping the server.
 -----------------------------------------------
 "Vulnerability" B:
 ==================
 v2.10.7 stops with a SIGPIPE because it lacks its (accidentally disabled) signal handler 
 to handle, ahem, signals.
 -----------------------------------------------
 Conclusion: 
 ===========
 Those facts have been documented and fixed on Oct. 8th by v2.10.8 (see the text above), 
 that is, two days before Fredrik Widlung wrote his "exploit".

 Fredrik denies having found inspiration in v2.10.8's documentation despite having been caught
 reading it on Oct. 8th (in gwan.ch logs); that was two days before he wrote his "exploit".

 Fredrik's "advisory" wording suggests that AFTER this "exploit" people should upgrade G-WAN
 to get the "fix".

 This is incorrectly implying that G-WAN was corrected because of what Fredrik insists to call
 "research": on gwan.ch, the flaw was publicly documented and a fix was posted two days before
 Fredrik Widlung wrote his v2.10.7 "exploit".
  
 Now, regarding the quality of Fredrik's other "findings", he skips "sudo ./gwan -d:www-data" 
 (documented in the PDF manual and in "./gwan -h") to incorrectly claim that:
 
   "The daemon does not limit privileges and actually runs all routines as 'root'"
 
 In the same vein, he wrote what he knew was plain lies since he had read v2.10.8's timeline 
 two days before he wrote his v2.10.7 "exploit":
 
   "The latest version has been silently updated on the site without even increasing the
    version number"
 
 Before concluding:
 
   "The dishonesty is remarkable."
 
 Indeed. But on which side it is the case remains disputable.

 This "exploit" targets a documented (and fixed) accidentally disabled signal handler.
 Without this timeline full-disclosure, the flaw would not have been exploited (2 days later).
 
 This "exploit" targets v2.10.6/7 (both posted the same day) and available for download less 
 than 24h. No prior or later version was affected.

 Therefore, upgrading AFTER Oct. 8th or for any other version than 2.10.6/7 is technically
 pointless. But that did not prevent Fredrik from posting his "advisory" on hundreds of
 "security" Web sites (none of which even tried to check the bogus claim before relaying it).

The less naive among us will remark that v2.10.6/7 (posted the same day) were severely humbling Nginx, Varnish and all others once again. Each time G-WAN gets a boost in performance or scalability, the F.U.D. and censorship machine flies to new stratospheric highs to rescue the less gifted. No wonder why their servers are weak in the tech area: doing the right thing requires other skills:

G-WAN has this very nice key value storage feature using wait-free algorithms. I would LOVE to see that implementation, this is top-notch stuff. Very few people are capable of correctly coding such a thing.
Thomas Porzelt, Lighttpd's Development Team

Not all Web server authors are crooks. How refreshing. It recalls me the good old days of C coders' attitude: aiming for higher goals, and respecting those pioneers who inspire us with great new ideas and code.


worker threads, weighttp, xbuf_vcat(), AFTER_READ handler

Oct
7
2011

Changes in G-WAN v2.10.7/Linux: Development Release

  • G-WAN now always uses all its worker threads for both static and dynamic contents (see weighttp's tests)
  • rewrote the ab.c ApacheBench wrapper to collect CPU/RAM statistics and add support for weighttp
  • the gwan executable renamed 'gwan_1' will run with one worker ('gwan_4' for 4 workers, and so on)
  • optimized further memory management to use even less memory during huge loads (side effect: speed)
  • added a few MIME types including *.c, *.h, *.php, *.py, *.jsp, *.aspx, *.fcs, *.amf (JohnnyOpCode, Arek)
  • added xbuf_vcat() to add an array of buffers to an xbuffer, just like the writev() call (thanks NilssonRio)
  • optimized (2x) the HTTP errors path which was significantly slowed-down in the recent past versions
  • prevented POST / PUT requests from trying ot use use pipelined content, clearing some client troubles
  • moved an HTTP conformance test after the AFTER_READ handler is triggered for TCP-servers (thanks Progamer)
  • fixed broken lingering close in recent versions due to a typo (symptom: incomplete downloads for slow clients)
  • optimized system C headers research for the Linux distributions/configurations which have trouble finding files quickly.

The choice to use all worker threads will make G-WAN look slower on (single-threaded) AB tests, but G-WAN will be much faster when facing SMP client test tools like Lighty's weighttp. With only multi-Core CPUs in production, the choice was easy (the Pentium 4 is 11 years old and its successor, in 2005, was a dual-Core).


Bug fix, clean up, KV flag, Hexadecimal Dumps

Sep
16
2011

Changes in G-WAN v2.9.16/Linux: Development Release

  • fixed a double-free issue causing crashes after huge memory loads (like loan benchmarks)
  • removed malloc debugging wrapper which reduced performances in the recent versions
  • added http_t.h_do_not_track for the new (optional) W3 Consortium "DNT:" HTTP header
  • added a KV_NO_UPDATE flag to make kv_add() fail to update an existing KV store entry (thanks Ersun):
  • 1
    2
    3
    4
    5
    6
    7
    8
    9
     kv_t store;
     kv_init(&store, "users", 0, 0, 0, 0);
    
     kv_item item;
     item.key = "pierre";
     item.klen = sizeof("pierre") - 1;
     item.val = "pierre@example.com";
     item.flags = KV_NO_UPDATE; // do not update an existing entry
     kv_add(&store, &item);     // return old/new entry, or NULL:out of memory
    

    Note that if you are not using the new KV_NO_UPDATE flag then you MUST setup item.flags = 0; the kv.c and kv_bench.c examples have been updated to reflect this new policy.

  • added a "%v" format to s_snprintf() and xbuf_xcat() to dump data in an "hexdump -C" like format:
      0: 5B 44 65 73 6B 74 6F 70  20 45 6E 74 72 79 5D 0A | [Desktop Entry].
     16: 56 65 72 73 69 6F 6E 3D  31 2E 30 0A 54 79 70 65 | Version=1.0.Type
     32: 3D 4C 69 6E 6B 0A 4E 61  6D 65 3D 45 78 61 6D 70 | =Link.Name=Examp

Instead of 0-padded hexadecimal offsets I used aligned decimal offsets with pretty thousands, and a more compact format (handy for frame dumps in log files or during debugging). The output buffer must be at least 5x larger than the input buffer. If the output buffer is too small then no output is returned and s_snprintf() returns the required output buffer length.

You can dump 16 bytes of binary data this way: "%16v" (if "%v" is used then a zero-terminated string is expected).


It's good to be back to the office: the last month was spent abroad, with only a day during week-ends available for coding - and the quality suffered. Focus is everything.


HTTP Pipelining, rotated log files date

Sep
4
2011

Changes in G-WAN v2.9.4/Linux: Development Release

  • fixed the rotated log file dates (since last version it used a boolean variable)
  • fixed the HTTP pipelining support (we should expect more requests per connections from new browsers).

Inline ASM, get_env(ROOT_PATH)

Aug
28
2011

Changes in G-WAN v2.8.28/Linux: Development Release

  • fixed the timestamp variable used to rotate the gwan.log file
  • added support for inline ASM in C scripts (see the asm.c example)
  • made WWW/CSP/LOGS/HLD_ROOT get_env() values use different pointers.

Bug Fixes

Aug
21
2011

Changes in G-WAN v2.8.21/Linux: Development Release

  • fixed a time/date stamps glitch in log files
  • fixed the directory listings mess (one line of code)
  • fixed the cache1,2,3.c examples and the cacheget() call
  • restored "application/octet-stream" as the default MIME type
  • added the "Accept-Language:" HTTP header to the http_t structure (see served_from.c).

Those little glitches were due to the whole v2.8 rewrite which has broken many minor things that worked fine in v2.1. Why rewrite the newborn G-WAN while veterans like Apache or Nginx don't bother? Well, G-WAN's design and implementation seek to be optimal – and this is what makes all the difference with the incumbents.


Bug Fixes

Aug
14
2011

Changes in G-WAN v2.8.14/Linux: Development Release

  • fixed the http->h_cookies issue (the HTTP header was incorrectly parsed)
  • modified the get.c servlet example to read as much data as made available
  • fixed the If-Modified-Since glitch (time comparisons were using different units).

The no-keep-alives problem was a by-product of the above time format issue. Like, probably, the abrupt connection cuts for visitors in Asia or Latin America (high latency).

Someone reported a problem with G-WAN's persistent pointers. I tested them all without a glitch in the persistent.c example. Note that handlers may access a connection state at which time the persistent pointer is not available YET (after accept(), the server does not know yet which virtual host is involved). As a result, you should always test the returned value before using it, or use the global G-WAN persistent pointer to avoid these cases.

I did not address the symbolic links reported issue, that will be for later. I believe that the fixes above are already worth having.

Thank you for the prompt feedback. This release candidate still needs a bit more love but we are on the right path!


More examples

Aug
13
2011

Changes in G-WAN v2.8.13/Linux: Development Release

  • added an email.c servlet example to illustrate the sendemail() call
  • added a persistence.c servlet example to illustrate G-WAN pointers
  • added a CLIENT_SOCKET value for get_env() to interact with clients
  • modified the post.c servlet example to read more data for long entities
  • modified the kv_bench.c servlet example to let it work without Tokyo Cabinet.

The gwan.ch web site has been updated with a more modern design, and with more features: you now can support G-WAN financially to make it evolve faster and stay alive on the long term!


"G-WAN v2.1 is a wimp" says G-WAN v2.8

Aug
8
2011

Changes in G-WAN v2.8.8/Linux: Development Release

  • a bunch of new optimizations lead to a nice 5-10x speedup
  • added an HTTP Headers struct for get_env(HTTP_HEADERS)
  • added the "%n" format to the s_snprintf() / xbuf_xcat() calls
  • added US_HANDLER_STATES to get_env() (thanks Atmo)
  • added SCRIPT_TMO/KALIVE_TMO to get_env() for timeouts
  • quoted ETag headers (thanks Marc Lehmann, libev's author)
  • added the cacheget() call for C scripts to find cached entries
  • replaced the cacheadd() data structure by a new faster beast
  • added the key-value store kv_add/_get/_del/_free/_do() calls
  • added the strerror() "%m" format in s_snprintf() / xbuf_xcat()
  • used a RDTSC fallback, tell me: I don't have the HW to test it
  • added listener/hosts details in the daily HTML server reports
  • added Garbage Collection: gc_malloc()/gc_free() for C scripts
  • added HDL_HTTP_ERRORS to let handlers redirect 404 errors
  • added HDL_AFTER_WRITE to let handlers do all (thanks Chang)
  • fixed non-existent paths invoked by handlers before parsing
  • allowed handlers to query the handler-folder before parsing
  • fixed split POST issue with IE and Chrome (Firefox was fine)
  • modified get_reply() so it no longer resets reply->len (Atmo)
  • the get.c example uses connect(), read(), write(), and close()
  • made Linux DNS lookup work with asynchronous socket calls
  • G-WAN/Linux now uses as many threads as CPUs/CPU Cores
    and full 64-bit timers (thank you Chang for the pertinent report).

By comparison, past versions were mere experimentation. This version pushes the multi-Core design to its long-term form (1 to 10,000-Core ready), brings the (wait-free) G-WAN Key-Value Store and a total rewrite of the Memory Manager (reducing the load, using garbage collection, getting better locality) and of the Lorenz Waterwheel (server and worker threads and traffic flow scheduler). This allows G-WAN to run faster in every possible case (single-Core and multi-Core CPUs, small and large payloads, low and high concurrencies) and even to outdo Nginx's exemplary memory usage (for static and dynamic contents).

OK, it took me 4 months but this new G-WAN puts previous versions to shame:

  • Old  G-WAN loan(100 years):   8,000 requests per second;
  • New G-WAN loan(100 years): 70,000 requests per second.

Proof that incremental enhancements and users' feedback have their value: nothing better kills a project than the belief that what has been done in the past should not be broken (as many times as needed) to reach the next step.

C scripts parsing error reporting has also been enhanced: Terminal ANSI codes effects make it clearer where the error is located in the C source code, and C scripts are renamed only when G-WAN runs as a daemon (without a programmer trying to fix the script). It was suggested a long time ago to get rid of the script ("script.c.bug") renaming feature, and this modification will make it less intrusive.

Tomek Gryczka also signaled that the error.css file was not used for resources located outside of the root folder. This is now fixed, like the large downloads (the variable size, a bit-field, was just too small) and the index.html file used in folders to prevent directory listings. Per the request of two G-WAN users, I have also added three MIME types "rdf+xml", "text/turtle" and "application/x-encrypted-gwan".

Also, you can now define which notifications a Handler's main() will receive:

1
2
3
4
5
6
8
9
10
11
12
13
14
int init(int argc, char *argv[])
{
   // get a pointer on the Handler states
   u32 *states = (u32*)get_env(argv, US_HANDLER_STATES, 0);

   // setup the Handler states that we want
   *states = (1 << HDL_AFTER_ACCEPT)
           | (1 << HDL_BEFORE_PARSE)
           | (1 << HDL_HTTP_ERRORS); // only those states

   return 0; // >= 0:success
}

Per the demand of Chang, I have added a server_report() call which lists the current state of G-WAN either in HTML or text (like in the daily HTML report), see the report.c example for details.

Linking libraries is now much easier since G-WAN will try to find the relevant path when not provided (you can still provide a full path if G-WAN does not find the library - in which case an error message is printed on the terminal):

1
2
3
4
5
6
7
8
9
// what you had to do in the past:
#ifdef LP64
# pragma link "/usr/lib32/libsqlite3.so.0" // 64-bit
#else
# pragma link "/usr/lib/libsqlite3.so.0"   // 32-bit
#endif

// what you can do from now:
#pragma link "sqlite3"

Finally, removing the G-WAN version number from the "Server:" HTTP response header, as suggested by Thomas Meitz, can only contribute to make G-WAN safer. I also documented /include/xbuffer.h for every call (thanks sNielsson) like is is the case for /include/gwan.h.

Thank you for all the great feedback on the G-WAN forum (which listed 1,500+ new registered users in only 6 months) - special thanks for Paco for having made it possible in the first place by graciously setting up the forum and hosting it!


Formatting, HTML redux

Jan
20
2011

Changes in G-WAN v2.1.20/Linux: Development Release

  • fixed the recently broken formatting of crash reports
  • fixed missing char when HTML files started with blanks
    (this was due to the on-the-fly HTML 'redux' feature).

Your feedback is improving G-WAN, thanks to 'Ascent' for the HTML bug. Still have to address the xbuf_frurl() client connection cut for slow servers.


Out of EPOLLution

Jan
19
2011

Changes in G-WAN v2.1.19/Linux: Development Release

  • finally got my way out the (nasty) EPOLLHUP trap.

The CPU usage is clean from kernel panics now, making G-WAN as fast as expected. Asynchronous BSD socket calls have now as little overhead as synchronous calls, making them suitable for all tasks (like G-WAN C script handlers - more on this soon).


csp scheduler, Chrome

Jan
18
2011

Changes in G-WAN v2.1.18/Linux: Development Release

  • added data URI padding for Google Chrome (IE/Firefox were fine)
  • fixed asynchronous calls calling the G-WAN server from C scripts
  • rewrote the C scripts scheduler completely (no more 'ab' errors).

Thank you all for the great feedback!


Auth, JSON, Ranges, POST

Jan
16
2011

Changes in G-WAN v2.1.16/Linux: Development Release

  • added binary application/octet-stream POST support (see post.c)
  • added an extensive JSON example to illustrate the 9 new functions
  • added support for the "If-Range:" HTTP header (byte ranges only)
  • added BASIC and DIGEST HTTP authorization schemes support
  • added a Data URI (icon) workaround for MS Internet Explorer 6/7
  • simplified the get_env() call while keeping backward compatibility*
  • fixed the conflicts between asynchronous blocking calls and epoll
  • fixed HTML redux: it turned "<p style" into "<pstyle" (thanks Mike)
  • fixed URL parsing that was eating encoded spaces (thanks Paco)
  • fixed the escape_html() function call broken unicode processing
  • fixed cases where the returned HTTP status code wasn't set (zero)
  • removed the duplicated entry at the bottom of the directory listings
  • added an HTTP code parameter to cacheadd(), see cache0.c and
    the manual (cache redirections: ret = 301, JSON payloads: ret = 1)
    MAKE SURE TO UPDATE SCRIPTS USING cacheadd().

Starting with 2011, G-WAN version numbers will use the YY.MM.DD format.

This version is much faster because it reduces the number of connections needed to serve the SAME contents (see 'Data URIs' in the manual). For example, the gwan.ch/index.html page is served with 4 times less requests, leading to lower latency and (4 times) better scalability.

The json.c example illustrates serialization and deserialization, how to search items by name, by value or by index, how to delete them, update them or add them, how to create arrays, etc. JSON RPC looks good. Correctly implemented, it will fly.

A note about BSD socket calls made asynchronous by G-WAN in C scripts: they work fine (see attack.c, post.c, or request.c, getheaders.c), but there are are still problems when you connect() to G-WAN from within a G-WAN C script (as opposed to reaching another server from a G-WAN script). Then, there are cases that trigger an EPOLLHUP tempest by the kernel. This is happening with relatively high concurrencies and I have not yet found how to get rid of this.

[*] char *entity = get_env(argv, ENTITY, 0); can now be used just like for integers: u32 port = get_env(argv, SERVER_PORT, 0); limiting the use of the last parameter (here zero) to the rare cases where we need to change a value (like DOWNLOAD_SPEED or HTTP_CODE). All the servlet examples have been updated with the new, simpler, get_env() possible usage.


JSON, CSS/JS/HTML, daemon, -r switch

Dec
16
2010

Changes in G-WAN v1.1.61/Linux: Development Release

  • added a JSON parser / renderer made available for C scripts
    (a dedicated example will come for Christmas)
  • added on-the-fly CSS/JS/HTML reduction (blanks, comments...)
  • added on-the-fly CSS Link URI to Data URI conversion for icons
  • removed the mlockall() call which makes the daemon mode fail
    (making "connection resets" happen unexpectedly, no clue why
    yet, but thanks a bunch to 'Scott' for signaling a problem)
  • added idiot-proof '-r' support for Tamás TEVESZ, fan of FUD.

Before 1.1.61, C servlets run with 'gwan -r' crashed because the server is not running (the manual states that the -r switch runs stand-alone general-purpose C source code rather than G-WAN C servlets). With the new dedicated G-WAN warning, even Tamás TEVESZ, a team member of the slower-than-Apache-1.3 Caudium web server, will no longer be able to shoot himself in the foot.

In the same vein, a "Web Hosting Guru" puzzled a G-WAN user by stating that "[G-WAN] benchmarks are misleading, you can achieve similar performance by writing modules for Apache, nGinx, Lighttpd etc. or by using libevent/evhttp directly".

False: because Apache, nginx, Lighttpd, libev, etc. are all slower than G-WAN for static contents and C scripts are almost as fast as GCC compiled modules, no force on Earth will make those servers faster than G-WAN for dynamic contents.

Even for heavy 3D (CPU intrinsics) computations, G-WAN C scripts could just use '#pragma link' to use an OpenGL library (supporting specific CPU instructions) to beat any (incomparably more complex) Apache or nginx pre-compiled module.

And G-WAN C scripts do it in 7 lines of code (without any stop or configuration):

1
2
3
4
5
6
7
#include "gwan.h" // G-WAN exported functions

int main(int argc, char *argv[])
{
   xbuf_cat(get_reply(argv), "Hello World");
   return 200; // return an HTTP status (200:'OK')
}

Compare that to the Apache and nginx 'hello world' modules which force you to:

  • create/edit the module C source code, (just like the shorter G-WAN C script)
  • + stop the (Apache or nginx) Web server,
  • + compile the module (Apache) or re-compile the Web server (nginx),
  • + edit the Web server configuration files for any newly created module,
  • + restart the Web server to use the newly created module.

You have to do this all before you can start testing your code. If there's a bug, or a feature that you want to enhance, then you have to do it all again.

With a G-WAN C script, you press F5 in the browser and the newly edited script is executed on-the-fly - considerably increasing your productivity.


Crash logs, Vary, set_reply()

Dec
6
2010

Changes in G-WAN v1.1.6/Linux: Development Release

  • removed the set_reply() call, see below to use the new get_reply():

    1
    2
    3
    4
    5
    6
    7
    8
     xbuf_t reply; get_reply(argv, &reply); // before
     xbuf_t *reply = get_reply(argv);       // now
    
     xbuf_cat(&reply, "Hello World!");      // before
     xbuf_cat(reply, "Hello World!");       // now
    
     set_reply(argv, &reply); return 200;   // before
     return 200;                            // now
    
  • added the full URL (with parameters) in C script crash reports
  • added the "Vary: Accept-Encoding" HTTP header for proxy servers
  • used parameterized queries to prevent SQL injection in sqlite.c
  • huge dynamic buffers (like loan 1,000 years) that were not freed
    because of a script timeout are now released.

Edit your servlets when they say: undefined symbol 'set_reply' to match the new get_reply() usage (see above) and remove all set_reply() calls.

In C scripts, set_reply() was sometimes missing before a return(), leading to nasty bugs. Using an xbuf_t pointer for get_reply() also makes it (much) easier for G-WAN to control the memory allocated by C scripts.

Thanks to Alex Gacovski for his feedback about the loan(999) issue. Because of floating point overflows, a 999-year loan loops forever (G-WAN detects this and timeouts), and that (rather rare condition) was potentially leaving unreleased buffers before v1.1.6.


Asynchronous BSD calls

Nov
23
2010

Changes in G-WAN v1.1.5/Linux: Development Release

  • added the Host Alias feature (map domain names to root host)
  • added syscall and libc symbol lookups in C script crash reports
  • added the C 'continuations' (request.c, getheaders.c, attack.c)
  • kill the daemon when gwan can't listen or when script is buggy
  • fixed an epoll issue with 'ghost' events raising the CPU usage
  • removed close() time-outs from log files (all the clients do that)
  • added ac email 'attachment' feature to the sendemail() function
    (I still have to test it and to document it).

The 'continuations' are really black-magic: they let you write procedural code in C scripts (G-WAN Servlets and Handlers) but work asynchronously behind the scene so G-WAN's threads are never blocking.

This is doing marvels with G-WAN sending requests to database servers, or other back-end servers (G-WAN used as a reverse-proxy to feed G-WAN or another application server/script engine).

And... it should work transparently with all the existing network libraries! (I only implemented the client-side since G-WAN does the server-side but I could add an asynchronous FILE SYSTEM support for all among you who are accessing remote file systems).

More will come soon about this new feature, with real-life examples.


Timeouts/logs/reports/buffers

Nov
12
2010

Changes in G-WAN v1.1.4/Linux: Production Release

  • added a 'grace' delay for 'interactive' (save as) downloads
  • added average traffic and gwan memory usage in HTML reports
  • added local time GMT+offset in the gwan.log file and HTML reports
  • tuned dynamic buffers to release memory faster than in the past.

Daemon, kill, DoS

Nov
2
2010

Changes in G-WAN v1.1.3/Linux: Development Release

  • fixed the PDF manual fonts mess made by Adobe Acrobat 7.0
    (now Windows users can read the Linux PDF manual)
  • added a message explaining why -k can fail (lack of privileges)
  • check for existence of the group / user profile specified with -k
  • added DOWNLOAD_SPEED env. variable to tweak the DoS shield.

Thanks to Alex Gacovski for the testing and user-friendly warnings suggestions and to Jym Cheong for the DoS (constructive) criticism!


daemon, kill, unicode

Oct
30
2010

Changes in G-WAN v1.1.2/Linux: Development Release

  • added a warning if -d is used without 'sudo' or the 'root' account
  • a readlink() failure prevented -k from finding pid files, patched it
  • restored unicode support in query strings (feedback is welcome).

Thanks to Alex Gacovski for the two first issues and to Jym Cheong for unicode!


Trace, performance counter, report

Oct
29
2010

Changes in G-WAN v1.1.1/Linux: Development Release

  • added a gwan/trace file to store the start/stop status of the process
  • added the parent and child uptimes for reports made in daemon mode
  • added times a (daemon) parent forked a child in reports and gwan.log
  • fixed a decimals error for the s_snprintf()/xbuf_xcat() "%llk" formater
  • fixed the inverted CC_BYTES_INDAY/OUTDAY perfomance counters.

Now G-WAN/Linux is keeping our most hysterical visitors at bay, I will have time to work on the SSL/TLS and C continuation features and deliver the SCGI and Reverse-Proxy example Handlers. A real-life 'forum' C application will follow to illustrate how to make it all work together.

And if someone knows how to make statvfs64() work on Linux v#1 SMP Thu Sep 16 15:56:38 UTC 2010 built 2.6.26-2-amd64 (ia32) then please let me know.


Sendemail, RESTFUL, timeouts, etc.

Oct
27
2010

Changes in G-WAN v1.1.0/Linux: Development Release

  • added sendemail() an SMTP client supporting 'auth login' for C script
  • added SHA1/SHA2 hashing and AES encryption functions for C script
  • added Linux 'Cpus_allowed' confinement messages in logs and reports
  • added list of network interfaces in gwan.log, help and error messages
  • added CPU load (1/5/15min history) and system uptime in logs/reports
  • added a "/csp/" alias of the "/csp?" prefix for RESTFUL Web services
  • enhanced the thread scheduling policy to shorten latency a bit further
  • enabled timeout Denial of Service attack logging in gwan error.log files
  • HTTP Keep-Alives no longer shortened to 2 seconds (a v1.0.97 glitch)
  • xbuf_frurl() now avoids infinite redirection loops from 'buggy' servers
  • extended the Common Log Format (CLF) to the Combined Log Format
  • applied stricter Unix permissions to the gwan/... files and folders
  • fixed a servlet renaming glitch (thanks to Alex Gacovski's bug report)
  • fixed the virtual hosts feature (thanks to Bob Stanton's bug report)
  • fixed -d:group:user (thanks to Anton Tameev and Alex Gacovski reports)
  • servlets can send JSON/XML/etc. bare contents (no HTTP headers or
    HTML body) by returning an invalid HTTP status code in the 1-99 range.

On Ubuntu 8.10 32-bit (Desktop), 'Cpus_allowed' (/proc/<pid>/status) restricts gwan to 2 Cores (unless you run 'sudo gwan'). G-WAN detects this now and reports it into the parent Terminal (if any), daily server report, and gwan.log file (letting you know why only one single physical Core may be involved: with hyper-threading, each physical Core counts for two logical Cores, and the Linux scheduler happens to jail a process into such a small environment even if, like me, you have 4 physical Cores begging for work).

I expected G-WAN v1.1 to provide SSL/TLS and continuations (the code is here, hence the extra bloat, but it is not used yet) allowing easy handler implementations like SCGI and Reverse-Proxy but the two last days kept me busy with nocturnal attacks on the new Linux server that hosts gwan.ch.


gwan.ch migrates to Linux!

Oct
20
2010

On October 11th, 16 of my domains were hijacked (like millions of other domains). It was clearly time to migrate my DNS and Web servers out of the reach of the MICROSOFT strategic partners (an environment where strange things happen routinely). Doing so has delayed the new G-WAN version, but G-WAN will also benefit from being used for production on a Linux 64-bit server. Good-bye Windows Server 2003, hello Linux.


Restored Connection Time-outs

Aug
20
2010

Changes in G-WAN v1.0.97/Linux: Development Release

  • restored adaptative time-outs (a glitch since v1.0.6 disabled them).

A lack of adaptative time-outs does not have consequences on performances (G-WAN time-outs can be disabled with the '-b' command-line switch) but it makes Web servers like Apache (300 seconds) or IIS (120 seconds 'after accept') vulnerable to denial of service attacks that open connections and send/receive data much too slowly (saturating your Web server's memory) or never send a complete HTTP request (saturating the system memory).

Apache and IIS neglect this deadly issue to the point where a single ADSL client can put down the so-called 'rock-solid' servers.

Slowloris is a breeze compared to MICROSOFT MSN/Bing robots and Cyveillance robots (a 'strategic partner') which are more agressive and sneaky (but this is not visible in Web server logs so nobody complains).

G-WAN uses adaptative time-outs (rather than fixed-size time-outs) to defeat all the possible cases. This mechanism allowed G-WAN/Windows to survive millions of such (duly logged) attacks since July 2009 -and it is now available in G-WAN/Linux (just in case, like me, you want to stay online).


#pragma link and #pragma include

Aug
7
2010

Changes in G-WAN v1.0.96/Linux: Development Release

  • added script support for '#pragma link' (for .c, .obj, .a/.lib, .so/.dll files)
  • added script support for '#pragma include' (for additional include paths)
  • created a gwan/include folder to store all the files shared by servlets
  • added servlet support for md5/crc32/adler32 and gzip/lzjb compressors
  • added log_err() to let servlets dump text in their virtual host's error.log
  • new examples: trace.c (tracing code) and sqlite.c (SQL database engine).

Many libraries are already installed in /usr/lib on Unix systems. Now, G-WAN lets you play with them by just using a couple of #pragmas (see the sqlite.c source code).

Who said that using C was about re-inventing the wheel?

Look at the XAMPP or PHP projects -and let G-WAN use the same C libraries to deliver features like databases (gdbm, SQLite, mySQL, PostgreSQL), XML (Expat, libxml), RegEx, JSON or XML parsing, PNG/JPEG/MPEG, VoIP, mCrypt, mHash, FreeType (TrueType fonts), fPDF and Ming (Flash), SMTP/POP3/IMAP, OpenLDAP, CURL, zZIP, etc.


Client IP fixed

Jul
28
2010

Changes in G-WAN v1.0.95/Linux (version unchanged, build date updated): Development Release

  • get_env(argv, REMOTE_ADDR, 0); reported a wrong IP address if
    any of the a.b.c.d items was > 199; that was because of a typo (a
    missing comma) and this is now corrected.

Thanks to 'Jason' for the bug report!


Set GID/UID, Charts and reading GIFs

Jul
27
2010

Changes in G-WAN v1.0.95/Linux: Development Release

  • gif_parse() (see the 'chart.c' example) parses in-memory GIF images
  • gif_build() (see the 'fractal.c' example) now supports GIF comments
  • added dr_chart() for bar/line/area/dot/pie/ring charts and sparklines
  • added dr_text() to print text in bitmaps using bitmap/TrueType fonts
  • added dr_line(), dr_circle(), dr_rect()/dr_recf() (handy to have them)
  • added dr_gradient() for multi-gradient palettes based on RGB values
  • added sw_rnd() a 2^158 PRNG, and hw_rnd() a fast Hardware RNG
  • added get_env() USER_AGENT and FNT_ROOT 'enviroment' variables
  • better documented G-WAN exported functions in the gwan.h header
  • headers (gwan.h, xbuffer.h...) are now stored once: in the gwan folder
  • get_env(argv, REMOTE_ADDR, 0); no longer gives a constant 0.0.0.0
  • cacheadd() no longer fails when called successively for a same entry
  • reduced even further the CPU load by replacing arithmetic routines
  • the '-d' command-line switch now lets you specify a 'group' / 'user' to
    dump 'root' privileges when using the daemon mode.

Productivity: C scripts allowed to develop the charts library rapidly. It was nice to write code and press 'F5' in the Internet browser to see the result (no wait with compilation/linking and no need to use a tool like GIMP to manually open every new GIF file made on-the-fly). Once it worked, the code was then moved into gwan (where, good news, it did not execute any faster).

G-WAN area/bar/dot/line/pie/ring charts

dr_chart() makes sparklines too (see the chart.c source code)

Coming next: LISP/SCHEME (true) 'continuations' and 'closures' - for ANSI C. Waving wait-states will speed-up client requests initiated by G-WAN (and C scripts) to back-end servers like database, email, proxy or other app. servers.


In-memory GIF I/O support

Jun
7
2010

Changes in G-WAN v1.0.9/Linux: Development Release

  • gif_build() (see the 'fractal.c' example) makes on-the-fly GIF images
  • xbuf_xcat() with "%B" encodes a string in base64 ("%-B" to decode)
  • improved source code line numbers lookup in C servlet crash reports
  • fixed: POST HTTP requests were improperly treated as HEAD requests.

gif_parse() will complete gif_build() to let us print text in bitmaps with bitmap fonts (made from TrueType fonts). Thanks to 'Dopey' for the bug report!


Minor Release

May
27
2010

Changes in G-WAN v1.0.8/Linux: Development Release

  • maintenance scalability: the impact of web sites file updates is now null
  • fixed issue: servlets updated with a bug can no longer raise a segfault.

Thanking 'SaltwaterC' for the very precise and relevant bug report!


Minor Release

May
20
2010

Changes in G-WAN v1.0.7/Linux: Development Release

  • made gwan work on virtualization layers (got memory allocation issues)
  • the '-d' and '-k' command line switches now work in every possible case
  • gwan no longer stops working after first servlet on a single-core system
  • corrected the CPU brand string in reports and logs (a cosmetic change)
  • xbuf_xcat(&buf, "%k", 1024); gives "1.00 KB" ("%llk" for 64-bit values).

Thank you for the feedback that made it possible to identify various issues!


"G-WAN 1.0.5 is a wimp" says G-WAN 1.0.6

May
14
2010

May 14th, 2010 • "G-WAN 1.0.5 is a wimp" says G-WAN 1.0.6

Changes in G-WAN v1.0.6/Linux Development Release (you *must* read the updated manual):

  • IP and domain-name based virtual hosts support (no configuration files)
  • implemented awaited handlers, the maintenance script, and persistence
  • added cacheadd() and cachedel() to let servlets store entries in cache
  • added getms() and getus() to get current time in millisec and microsec
  • added gzip+deflate on-the-fly compression (server replies >100 bytes)
  • extended get_env() to access internal performance options/counters
  • added a '-b' command-line switch to disable the Denial of Service shield
  • added a '-d' command-line switch for a daemon mode (angel process)
  • added a '-k' command-line switch to gracefully stop gwan (daemon)
  • added a '-r' command-line switch to run C scripts (like the ab.c test)
  • escape_html() uses unicode for 8-bit chars and respects word breaks
  • xbuf_xcat() supports the ' pretty thousands (%D, %U, %F still work)
  • xbuf_xcat() supports the '#' flag (decimal point with no decimals)
  • xbuf_xcat() supports %b to print 16-bit, 32-bit and 64-bit binary ints
  • servlets life is now limited by a time-out value (no more infinite loops)
  • servlets that fail return '500:Internal Error' rather than '404:Not Found'
  • fixed: files deleted on disk can no longer return the next cache entry
  • fixed: removed a stupid limitation for long servlet output (loan100+)
  • fixed: I have had a hard time to identify/fix this system socket issue
  • the ab.c benchmark framework now dumps HTTP errors (IIS locks)
  • factor 4 speed gain (the kernel-bottleneck will let you see only 2% of it)
  • factor 5 scalability gain (hopefully, you will see 100% of this clever one)
  • factor 20 (peak value) of CPU resources savings (high-concurrencies)
  • high-concurrency attacks on long-lasting scripts are no longer harmful.

Upgrading G-WAN/Windows to v1.0.6 would not make it as fast as on Linux: the Windows kernel has hit the wall (see notes [2] & [3] and Linux vs. Windows). As long as Microsoft does not fix its kernel, it does not make sense for TrustLeap to invest more time on Windows because TrustLeap's goal is to offer the most efficient solution for the benefit of developers and end-users.


TrustLeap's Forum is available

Dec
19
2009

The long-awaited G-WAN Forum is now online, thanks to JohnnyOpcode!


G-WAN Linux/Win32 features now on par

Dec
4
2009

Notes regarding G-WAN v1.0.5/Linux: Development Release

  • C scripts SIGSEGV/SIGFPE no longer stop G-WAN -they log a crash dump
  • tuned the 'Lorenz waterwheel'/Linux (giving more scalability).

To come soon: connection handlers, a maintenance script, and library examples.


G-WAN/Linux publicly released!

Nov
30
2009

G-WAN is MUCH faster on Linux than on Windows.

Notes regarding G-WAN v1.0.5/Linux: Development Release

  • C scripts SIGSEGV and SIGFPE not (yet) blocked -this is a bit tricky
  • timeouts are not fully tested -but they work at least for trivial tests
  • far-too-slow and idle-for-too-long connections are properly closed.

Notes regarding G-WAN v1.0.5 on Windows: Development Release

  • added the -b command-line switch: no DoS shield (for benchmarks)
  • a bit of code-cleaning took place (thanks Linux), more to come soon.

Next step: Solaris. After that, G-WAN's serious features will be implemented.


Minor Update

Sep
9
2009

Changes in G-WAN v1.0.4/Windows: Production Release

  • removed more branching, resulting in more speed (yes, again)
  • fixed duplicate timeouts on Windows Server 2003/2008 platforms
  • helped to save Windows from itself with long-lasting TCP connections

G-WAN is faster and faster – despite more features!


Denial of Service Shield

Aug
26
2009

Changes in G-WAN v1.0.3/Windows: Development Release

  • enhanced the 'Lorenz waterwheel' (even more speed & scalability)
  • removed more branching, resulting in even more speed than before
  • implemented an 'anti-flooding' check, protecting system resources
  • finally found how to make timeouts work with IO completion ports
  • (now idle and [much too] slow connections are cut when needed).

Keep tuned!


One Month and half

Aug
15
2009

Changes in G-WAN v1.0.2/Windows: Development Release

  • implemented a 'Lorenz waterwheel'-inspired logic for scalability & speed
  • added support for real-time cache updates -without speed penalty
  • (no more G-WAN stop/restart to reload updated docs/servlets)
  • added support for the If-Not-Match header field (ETag's companion)
  • different log files are created for each new day, keeping files smaller
  • local time offsets in the access.log file time-stamps use daylight savings
  • too short requests trigger 400:Bad Request rather than 404:Not Found
  • restored support for absolute URIs (disabled by too stringent checks).

Thank you for your feedback!


First Week

Jul
7
2009

Changes in G-WAN v1.0.1/Windows: Development Release

  • added the missing 'errors.css' file to the /www/imgs sub-folder
  • added support for the (requested) response headers' ETag field
  • fixed a couple of gwan/access/error log files format glitches;

More to come soon!