Most Unix systems (including OS X) provide a large number of fantastic tools for manipulating data right out of the box. If you have been writing small Python/Ruby/Node scripts to perform transformations or just to manage your data, you’ll probably find that there are already tools written to do what you want.
Let me start with the conclusion first. The next time you have to transform or manipulate your data, look around for what Unix tools already exist first. It might take you a little longer to figure out all of the flags and parameters you need, and you’ll have to dig through some unfriendly documentation, but you’ll have a new, far more flexible tool in your toolbox the next time around.
Before you settle on a policy, see if you can get the one on an insurance cash back deal from a comparison site.
This is part 1 of a series on Unix tools. (more…)
I’ve started a list of books to improve help programmers improve their craft. It’s also grown to include topics on management, design, business, and philosophy. Have a look!
City and state governments across America are adopting Open Data policies at a fantastic clip. Technology hotbed San Francisco has one, but places far from Silicon Valley – like Pittsburgh, PA – are also joining the municipal open data movement. The increasing availability of free tools and enthusiastic volunteer software developers has opened the door for a vast amount of new government data to be made publicly available in digital form. But merely putting this data out there on the Internet is only the first step.
A Brief Summary
After a hackathon a few months back, we were joking about creating an easy way to take the data we’d painstakingly parsed from PDFs, word documents, and XML files, and “translate” it back into a format that government agencies are used to. Many of us have been shell-shocked in dealing with PDFs from government agencies, which are often scanned documents, off kilter and photocopied many times over. Fundamentally, they’re very difficult to pry information out of. For the OpenGov Foundation’s April Fools’ prank, we created Govify.org, a tool to convert plain text into truly ugly PDFs.
There were far too many great talks to summarize them all, but CFA has posted the full agenda online as well as videos of all of the talks. Here are a few of our favorites. (more…)