* * * * *
Musings on the Current Work Project Du jour
So I have this Lua [1] code that implements the cellphone end of a protocol
used in “Project: Wolowizard.” I need to ramp up the load testing on this
portion of the project so I'm looking at what I have and trying to figure out
how to approach this project.
The protocol itself is rather simple—only a few messages are defined and the
code is rather straightforward. It looks something like:
> -- Pre-define these
> state_receive = function(phone,socket) end
> state_msg1 = function(phone,socket,remote,msg) end
> state_msg2 = function(phone,socket,remote,msg) end
>
> -- Now the code
>
> state_receive = function(phone,socket)
> local remote,packet,err = socket:read()
> if err ~= 0 then
> syslog('err',string.format("error reading socket: %s",errno[err]))
> return state_receive(phone,socket)
> end
>
> local msg,err = sooperseekritprotocol.decode(packet)
> if err ~= 0 then
> syslog('err',string.format("error decoding: %s",decoderror(err))
> return state_receive(phone,socket)
> end
>
> if msg.type == 'MSG1" then
> return state_msg1(phone,socket,remote,msg)
> elseif msg.type == "MSG2" then
> return state_msg2(phone,socket,remote,msg)
> else
> syslog('warn',string.format("unknown message: %s",msg.type))
> return state_receive(phone,socket)
> end
> end
>
> state_msg1 = function(phone,socket,remote,msg)
> local reply = ... -- code to handle this msg
> local packet = sooperseekritprotocol.encode(reply)
> socket:write(remote,packet)
> return state_receive(phone,socket)
> end
>
> state_msg2 = function(phone,socket,remote,msg)
> local reply = ... -- code to andle this msg
> local packet = sooperseekritprotocol.encode(reply)
> socket:write(remote,packet)
> return state_receive(phone,socket)
> end
>
Don't worry about this code blowing out the call stack—Lua optimizes tail
calls [2] and these effectively become GOTOs. I found this feature to be very
useful in writing protocol handlers since (in my opinion) it makes the state
machine [3] rather explicit.
Now, to speed this up, I could translate this to C. As I wrote the Lua
modules for The Kitchen Sink Lua interpreter [4], I pretty much followed a bi
level approach. I have a C interface (to be used by C code) which is then
mimicked in Lua. This makes translating the Lua code into C more or less
straightforward (with a bit more typing because of variable declarations and
what not).
But here, I can't rely on the C compiler to optimize tail calls (GCC can, but
only with certain options; I don't know about the Solaris C compiler). I
could have the routines return the next function to call and use a loop:
> while((statef = (*statef)(phone,sock,&remote,&msg) != NULL)
> /* the whole state machine is run in the previous line;
>
But just try to define the type of statef so the compiler doesn't complain
about a type mismatch. It needs to define a function that takes blah and
returns a function that takes blah and returns a function that takes blah and
returns a function that … It's one of those recurisive type definitions that
produce headaches when you think too much about it.
Okay, so instead, let's just have a function that returns a simple integer
value that represents the next state. That's easier to define and the main
driving loop isn't that bad:
> while(state != DONE)
> {
> switch(state)
> {
> case RECEIVE: state = state_receive(phone,socket,&remote,&msg); break;
> case MSG1: state = state_msg1(phone,socket,&remote,&msg); break;
> case MSG2: state = state_msg2(phone,socket,&remote,&msg); break;
> default: assert(0); break;
> }
> }
>
Okay, with that out of the way, we can start writing the C code.
Clackity-clackity-clack clackity-clack clack clack clackity-clackity-clackity
clack clack clack clack clack …
Man, that's boring drudgework. Okay, let's just use the Lua code and maybe
throw some additional threads at this. I don't think that's a bad approach.
Now, Lua, out of the box, isn't exactly thread-safe. Sure, you can provide an
implemention of lua_lock() and lua_unlock() but that might slow Lua down
quite a bit (there are 62 locations where the lock could be taken in the Lua
engine). We could give each thread its own Lua state—how bad could that be?
How big is a Lua state? Let's find out, shall we?
> #include <stdio.h>
> #include <stdlib.h>
> #include <lua.h>
> #include <lauxlib.h>
>
> int main(void)
> {
> lua_State *L;
>
> L = luaL_newstate();
> if (L == NULL)
> {
> perror("luaL_newstate()");
> return EXIT_FAILURE;
> }
>
> printf("%d\n",lua_gc(L,LUA_GCCOUNT,0) * 1024);
> lua_close(L);
> return EXIT_SUCCESS;
> }
>
When compiled and run, this returns 2048, the amount of memory used in an
empty Lua state. That's not bad at all, but that's an empty state. What about
a more useful state, like the one you get when you run the stock Lua
interpreter?
> -- ensure any accumulated garbage is reclaimed
> collectgarbage('collect')
> collectgarbage('collect')
> collectgarbage('collect')
> print(collectgarbage('count') * 1024)
>
Okay, when I run this, I get 17608. Eh … it's not that bad per thread (and I
do have to remind myself—this is not running on my Color Computer [5] with
16,384 bytes of memory). But I'm not running the stock Lua interpreter, I'm
running the Kitchen Sink Lua with all the trimmings—how big is that state?
I run the above Lua code and I get 4683963.
Four and a half megs!
Ouch.
I suppose if it becomes an issue, I could always go back to writing C …
[1]
http://www.lua.org/
[2]
http://c2.com/cgi/wiki?TailCallOptimization
[3]
http://en.wikipedia.org/wiki/Finite-state_machine
[4]
gopher://gopher.conman.org/0Phlog:2013/03/22.1
[5]
http://en.wikipedia.org/wiki/TRS-80_Color_Computer
Email author at
[email protected]
