Nim Day 3: IO
============================================================
At the end of Nim Day 2, I said I'd be talkin' about Nim's type
system next. But I did a little thinking since then and I've
decided to cover Nim's IO (input/output) facilities instead. The
change is for one simple reason: reading and writing tends to be
more fun!
When in doubt, I'd rather err on the side of having too much fun
with this visit to Nim-land than too little. And when I say
"fun", I mean getting into the meat of making things work rather
than worrying overly much about syntax. To be sure, I take syntax
very seriously and learning the fine details of a language is
important! But I've also learned that you can learn a *lot* of
minutia about a language's details and not ever learn *how to use
it*.
That's where I was years ago when read Stroustrup's _The C++
Programming Language_ cover-to-cover, did all of the exercises,
bought the STL reference book (some big Addison-Wesley hardcover)
and read through that, and then daydreamed about all of the things
I would build...and somehow never got around to actually using it
to build anything! What a waste!
The point is, I try not to repeat that experience. I still forget
as much as I learn, but I try to make sure that I can apply most
of what I'm learning.
File IO in the `system` module
------------------------------------------------------------
Let's jump right in and see what Nim provides in the core standard
library. The 'system' module contains basic IO facilities. It's
already imported into every Nim project, so there's no need to
include it. You can see the entire list of contents of the
'system' module here:
https://nim-lang.org/docs/system.html
The very first IO thing we run into in the list of methods is
`open()`. I also see a `writeLine()` and `close()` method further
down. Let's see if we write a file and read it:
# write.nim
var myfile: File
discard open(myfile, "hotdata", fmWrite)
myfile.writeLine("wigglers=3")
myfile.writeLine("danglers=6")
close(myfile)
A quick explanatory list:
- Create variable of type File to hold our file reference
- Call open() with the file referece, file name to open, and the
mode
- The file mode fmWrite means...oh, I think you can guess
- Nim requires that we do something with all return values -
`discard` is a non-action we can perform if we don't want the
value
- Call the writeLine() method on myfile and send it a string
- Do it again
- Close the file
- Don't bother with error checking because that's for wimps
Now we compile and run:
$ nim compile write
$ ./write
$
Exciting? Maybe, let's see if our file was written:
$ ls -l hotdata
-rw-r--r-- 1 dave users 22 Aug 27 18:35 hotdata
Yes.
$ file hotdata
hotdata: ASCII text
Yes!
$ wc hotdata
2 2 22 hotdata
YES!
$ cat hotdata
wigglers=3
danglers=6
Yahooo! Yessss, yessss, YES!
Okay, okay, calm down. I'm excited too. But can we *read* the
file?
# read.nim
var myfile: File
discard open(myfile, "hotdata", fmRead)
echo myfile.readLine()
echo myfile.readLine()
close(myfile)
By the way, there's a shortcut for compiling and running a Nim
program all at once.
$ nim compile --run read
Or shorter:
$ nim c --run read
Or my favorite:
$ nim c -r read
...
wigglers=3
danglers=6
Woo! There's our data.
Also, I find the Nim compiler to be a bit noisy by default, so we
can quiet it down like so:
$ nim c -r --verbosity:0 read
Now how about reading from STDIN and writing to STDOUT?
The `system` module defines those as File objects, so let's see if
we can play with those:
# stdio.nim
var foo = stdin.readLine()
stdout.writeLine(foo)
Boy, since these objects are already defined and opened for us,
this is sure a small program! Now we compile and run:
nim c -r stdio
...
Hey there, Gophersphere!!!
Hey there, Gophersphere!!!
Sweet, it echoes the line we typed and then exits. It's like a
terrible `cat`.
Redirecting stdin from a file works great, too, so this is the
real deal:
$ ./stdio < hotdata
wigglers=3
Neat!
Command line params in the `os` module
------------------------------------------------------------
If we import the `os` module, we can do some other handy things to
interface with the outside world. I guess these aren't what we
might think of as traditional "I/O" per se, but they do let us
"input" and "output". For example, we can get command-line
arguments ("arguably" a very important type of input) :
# args.nim
import os
echo "Argh! You gave me:"
echo paramStr(1)
We can pass arguments to programs through the Nim compiler's
arguments:
$ nim c -r args Foo
...
Argh! You gave me:
Foo
If we don't give our program, `args`, an argument, it will crash
(gracefully!) with an invalid index error:
$ ./args
Argh! You gave me:
Traceback (most recent call last)
args.nim(4) args
os.nim(1466) paramStr
Error: unhandled exception: invalid index [IndexError]
The `os` module also lets us examine files and directories, copy,
move, and delete files, execute external commands and that sort of
thing.
Streams
------------------------------------------------------------
For more advanced file IO, there is the `streams` module, which
has a ton of methods for reading and writing very specific amounts
and types of data to File 'streams'.
Let's just see a quick example of writing some binary data, just a
couple numbers:
# writebin.nim
import streams
var a: uint8 = 4
var b: int16 = 4095
let fs = newFileStream("hotdata2", fmWrite)
fs.write(a)
fs.write(b)
fs.close
Running this should create a binary file. Let's see:
$ nim c -r writebin.nim
...
$ hexdump hotdata2
0000000 ff04 000f 0000
0000005
Okay, that *looks* roughly right. At least there's a 4 in there
and 4095 is 'fff' in hexadecimal, so that's probably right, but
it's hard to tell when you factor in the default byte grouping,
endianess, etc. So let's just see if Nim can read this back in
for us!
# readbin.nim
import streams
let fs = newFileStream("hotdata2")
let a = fs.readUint8()
let b = fs.readInt16()
echo a
echo b
fs.close
And here's the results...
$ nim c -r readbin
...
4
4095
Hey, there are our numbers!
Strings
------------------------------------------------------------
This doesn't have anything to do with IO, but I want to use this
feature later and we need a break after reading and writing binary
data, don't we?
Nim's string handling is really excellent. I'm not going to
attempt any comparisons with, say, Perl, Tcl, or Ruby. But I
think it holds its own.
Certainly, in the world of compiled languages, Nim's string
support is very good. I think that's extremely important. I
don't know about you, but almost everything I do involves working
with strings in some form or another.
Here are three important features:
1) Concatenation is done with the `&` operator:
# stringcat.nim
let who = "Gophers"
echo "Hello " & who & "!"
Does what you'd expect:
$ nim -r c strings
...
Hello Gophers!
2) Conversion to string is done with the `$` operator:
# tostring.nim
let foo = 5
let bar = ['a', 'b']
echo "My Stuff! foo=" & $foo & ", bar=" & $bar
Quite nice output including a textual representation of the array:
$ nim c -r tostring.nim
...
My Stuff! foo=5, bar=['a', 'b']
3) Interpolation with `%` requires the `strutils` library:
# interpolation.nim
import strutils
let animal = "cow"
let beans = 4342088
echo "Oh no! The $1 ate $2 beans!" % [animal, $beans]
Prints:
$ nim c -r interpolation
...
Oh no! The cow ate 4342088 beans!
By the way, when you include the `strutils` library, it brings
with it a number of other libraries:
Hint: system [Processing]
Hint: interpolation [Processing]
Hint: strutils [Processing]
Hint: parseutils [Processing]
Hint: math [Processing]
Hint: algorithm [Processing]
But even though it all gets compiled in, it barely makes a
difference in the resulting executable and your program remains
fast, compact, and dependency-free.
Parsers
------------------------------------------------------------
On the subject of IO (particularly Input), I think it's highly
relavent to make note of the large selection of parsers that Nim
includes in the standard library. Here's a couple that might
pique your interest:
- `parseopt` - parser for short (POSIX-style) and long (GNU-style)
command line options
- `parsecfg` - a superset of ".ini"-style config files with
multi-line strings, etc.
- `parsexml` - simple XML parser - lenient enough to parse typical
HTML
- `parsecsv` - CSV file parser
- `json` - JSON parser
- `rst` - reStructuredText parser
- `sexp` - S-expression parser
There is also a whole other section devoted to multiple libraries
for parsing and generating XML and HTML.
The other thing I love about the Nim standard library is that it
includes lower-level tools you can use to implement other parsers
such a Perl-compatible Regular Expressions (PCRE) in the `re`
library, Parsing Expression Grammars (PEGs) in `pegs` and
lexers/parsers in `lexbase`, `parseutils`, and `strscans`. Those
are rich enough to warrant a post (or two or three) of their own.
But I doubt I want to be that thorough in this rambling phlog.
Rather than pick one of the parser libraries at random, I'm going
to demonstrate the one I used in real life just today to wrangle
some data.
The problem: I needed to extract a particular column from a CSV
file, perform a number of alterations to it, and then write it
back out (along with some other changes which are of no concern).
Let's pretend I just need to make one of the columns UPPER CASE.
First, our data file: (I've aligned the columns here so it's
easier to read.)
Category, Acronym, Definition
chat, lol, Laughing Out Loud
chat, brb, Be Right Back
sf, tardis, Time and Relative Dimensions In Space
nasa, sfc, Specific Fuel Consumption
nasa, acrv, Assured Crew Return Vehicle
Now our program:
# acro.nim
import os, parsecsv, strutils
var csv: CsvParser
csv.open(paramStr(1))
csv.readHeaderRow()
while csv.readRow():
echo "$1,$2,$3" % [
csv.rowEntry("Category"),
csv.rowEntry("Acronym").toUpper,
csv.rowEntry("Definition")]
csv.close
Now we cross our fingers and run to see if we were able to write a
good CSV: (Again, I've aligned the columns for our viewing
pleasure, but normally CSV data is packed,like,this.)
$ nim c -r acro.nim acronyms.csv
...
Category, Acronym, Definition
chat, LOL, Laughing Out Loud
chat, BRB, Be Right Back
sf, TARDIS, Time and Relative Dimensions In Space
nasa, SFC, Specific Fuel Consumption
nasa, ACRV, Assured Crew Return Vehicle
Awesome! The Acronym column has been converted to uppercase.
As I said, I actually did something very similar to this today. I
started off with AWK, which certainly made easy work of parsing
the data. But before I knew it, I was off in the weeds with an
AWK script that was getting towards twenty lines and I was
*fighting it*.
This is where you turn to your favorite "scripting" language such
as Python, Perl, or Ruby (I like all three). But with Nim, you
can write the program just as quickly *and* end up with a fast,
portable executable! I think it's just fantastic.
Okay, this post is in danger of getting crazy long, but there is
just one other "IO" related thing I want to mention...
Networking
------------------------------------------------------------
Nim's standard library includes libraries for CGI, an HTTP server,
an HTTP client, an SMTP client, an FTP client, and two levels of
socket support. So it's ready to create all sorts of interesting
applications right out of the box.
(Sorry, no Gopher libraries in the Nim standard library.)
Let's just see a really simple HTTP server example and call it a
day.
# trollserver.nim
import asynchttpserver, asyncdispatch
var troll = newAsyncHttpServer()
proc handler(req: Request) {.async.} =
await req.respond(Http200, "Go use Gopher instead!\n")
waitFor troll.serve(Port(8787), handler)
Now we compile and run:
$ nim c -r trollserver
And our server is waiting for us. In another terminal (or
full-fledged browser, for that matter) we can make a request:
$ curl localhost:8787
Go use Gopher instead!
$ curl localhost:8787
Go use Gopher instead!
What a ridiculous troll! But you can see how easy it would be to
make a little web interface to serve up some data with Nim.
I'll try to make the next installment shorter and perhaps a bit
more focused.
Until the next one, happy Nim hacking!
