Out of Hanwell

January 11, 2009

Basic Web Developer Skills

Filed under: Browsers, Development, JavaScript, Software, Web — Tags: — Matthias Miller @ 12:25 pm

The past several years at work, I’ve done a lot of HTML/JavaScript development. Last summer I handed off my work to another lead developer. During that process, my manager and I discussed the basic skills required for a good web developer, and we came up with this list.

Test cases. A good web developer turns problems into simple, reproducible test cases. When something goes wrong, it’s easy to fall into the lets-see-what-this-does rut. This solves your immediate problem, but it doesn’t leave you much smarter for next time. If you whittle down your problem to a comprehensible size, you can understand exactly what is going on, why it’s not working, and what you need to change. (Well, okay, some browser bugs are simply weird, but at least you have got a fighting chance!)

Research. A good web developer researches problems. You’re the smartest man alive, but you’re not the first to find this bug. You need to know which sites to check, such as QuirksMode, MSDN, MDC, and now Stack Overflow. But you also need to know how to use a search engine. Developers are creative creatures and can find a hundred ways to describe the same problem. You need to learn which keywords will bring the gold to the surface.

Documentation. A good web developer writes clear documentation. Once you solve a problem, it’s easy to check it in and move on. But just because you understand the problem today doesn’t mean that your team understands it or that you’ll understand it tomorrow. Write it down!

Localized Hacks. A good web developer favors localized hacks. Some problems can only be solved by ugly fixes, but you can minimize the damage by localizing the complexity. Keep the knowledge of that hack in one place–for example, hide it behind a JavaScript function, stick it into a CSS block, or fix it by adding a single element on your HTML page.

Pragmatism. A good web developer is pragmatic, neither sailing on the clouds of optimistic idealism or sinking to the depths of pessimistic despair, but somehow maintaining enthusiasm and courage in spite of the challenge.

Our conclusion? A good web developer is primarily a problem solver.

Really, it’s not complicated.


August 16, 2008

Khoros: A Lua Operating System

Filed under: Development, Linux, My Projects — Tags: , , , , — Matthias Miller @ 4:06 pm

Over the past year or two, I’ve been tinkering with my own operating system, which I’ve dubbed Khoros. The name comes from the Greek word χορός, meaning dance, from which we get our English words chorus and choreography. Of course, I chose this name because of its obvious play on words.

My first task was simply getting a bootable system. Rather than write my own bootloader, I used Grub. I based my work off Brandon Friesen’s excellent tutorial. I initially used VMware for virtualization, but since then, I’ve since switched VirtualBox and have had much better success.

As much as possible, the kernel is written in Lua. (Yes, Microsoft’s Singularity project caught my attention.) I am not particularly gung-ho over Lua as a language, but I picked it for its exceptionally small footprint, and it opens interesting possibilities for sandboxing. For example, my C kernel interfaces high-level interfaces to allocate and manipulate memory. However, the Lua scripts cannot specifically address memory. They can only manipulate memory block within the allocated bounds. This allows very powerful but safe low-level programming for my drivers.

With the past several weeks, I finished up basic multi-process support using Lua’s coroutines and debug hooks. (I am not using traditional process boundaries, so everything is running in kernel mode.) Processes can communicate with each other using a very simple IPC mechanism.

Today, my operating system boots into a simple Lua shell, allowing me to run arbitrary Lua code from the command line. This is no small feat, but it is admittedly of limited use. I am trying to decide how to proceed.

A networking stack could be quite beneficial, since it would allow me to program it remotely or to do something interesting like serve websites. However, this involves learning about the PCI bus, the NIC, ethernet, TCP/IP, DHCP, and many more details of which I am certainly unaware.

Alternatively, I could implement rudimentary file system support. For example, I could provide a way to save Lua objects to disk. However, I am lost to what interesting piece this might lead me to.

I blog so infrequently that I’ve probably lost most of my readership, but to all of you out there, what do you think I should develop next?

September 20, 2006

Automated Tests for JavaScript Lint

Filed under: Development — Matthias Miller @ 8:41 pm

Writing software is all about managing constraints. If you’re the only developer on a project, you need a development process that can thrive despite limited developer resources. Users will inevitably scream out and you won’t always be able to answer immediately. Can you respond in such a way that they feel heard, even though you cannot immediately solve their problem? When you do respond, how do you ensure that you’ve got a solution that works, while spending as little time as necessary?

Fortunately, small projects can be incredibly agile. In the early days of JavaScript Lint, I would release a new version for every bug fix reported by a user, sometimes with an hour or two of the report. However, these releases also sometimes introduced regressions that should have been caught by testing, requiring more busywork to release another version.

About a month ago I started looking for inexpensive and unobtrusive ways to test JavaScript Lint. Originally, I anticipated adding complicated control comments to supress warnings. These control comments would themselves raise warnings if the expected warning wasn’t encountered. However, I finally decided that I’d be better off writing a simple Perl script (not an oxymoron!). This script easily lets me tweak configuration settings and check for specific warnings and errors in test scripts.

I’ve built up a collection of test cases for all the warnings implemented in JavaScript Lint, as well as for some warnings implemented in SpiderMonkey. When a user reports a bug, I can simply create a new test case and develop against it until it passes. I’ve been surprised how much this has increased my productivity in the little time I’ve devoted to JavaScript Lint in the last several weeks. The time spent writing automated tests has undoubtedly paid off already.

Now, if I only have a chance to clean up those release scripts, perhaps I can roll out a new version of JavaScript Lint!

June 12, 2006

Big Problems

Filed under: Development — Matthias Miller @ 6:10 am

Have you ever felt like this? It reminds me of the window.onload problem.

Big problems are terrifying. There’s an almost physical pain in facing them. It’s like having a vacuum cleaner hooked up to your imagination. All your initial ideas get sucked out immediately, and you don’t have any more, and yet the vacuum cleaner is still sucking.

(From How to Do What You Love by Paul Graham)

May 13, 2006

SVG Icons

Filed under: Development — Matthias Miller @ 11:47 am

I recently used Inkscape to design an SVG logo for that development tool I’ve been talking about. Perhaps my inexperience explains why I’m so easily impressed by vector graphics, but here’s what I liked:

  • Attribution. I can attribute certain parts of the image that are derived from another source.
  • Accessibility. The image can be rendered at any required size and could (theoretically) degrade nicely for media with low color depth, should the software support it.
  • Consistent file size. The size at which the image is rendered doesn’t matter; it maintains a fixed file size. You might still end up with a huge SVG if you’re doing complex graphics, though.
  • Human-readable format. XML probably isn’t perfectly human readable, but compare it with JPEGs, GIFs, and PNGs and you’ll see what I mean.

As I started creating icons from the logo, however, I found that vector graphics are not integrated into the development process very well at all. Mac was pretty easy–create four PNGs and drop them onto the image tool–but Windows was a pain. Windows icons have five or six standard sizes and five different color depths, which means a lot of image conversion. Worse still, Microsoft Visual C++ does not support 256 color icons, so I had to hunt for an editor that did. Even then, I compromised. Any users with monochrome or 16 color displays will just have to suffer with whatever Windows gives them. (I apologize in advance!)

This made me wish for a one-click web service that creates icon files from vector graphics. It shouldn’t be so hard to do, but it would be very convenient. Eventually, of course, the service could support low color depths by removing gradients and by picking appropriate replacement colors.

May 1, 2006

Linking MSVC Libraries with MinGW Projects

Filed under: Development — Matthias Miller @ 11:47 am

Over the past several weeks, I’ve been reading Karl Fogel’s book Producing Open Source Software, having first heard about it from Ben Goodger. It’s full of practical advice for anybody’s who’s involved with open source software. But I’d like to focus on the advice about starting a project:

That sounds obvious, but many projects don’t bother to standardize their installation procedures until very late in the game, telling themselves they can do it any time: “We’ll sort all that stuff out when the code is closer to being ready.” What they don’t realize is that by putting off the boring work of finishing the build and installation procedures, they are actually making the code take longer to get ready—because they discourage developers who might otherwise have contributed to the code. Most insidiously, they don’t know they’re losing all those developers, because the process is an accumulation of non-events: someone visits a web site, downloads the software, tries to build it, fails, gives up and goes away. Who will ever know it happened, except the person themselves? No one working on the project will realize that someone’s interest and good will have been silently squandered.

I’ve been working with another developer on the software development tool that I recently mentioned: Compilation on the Mac was a breeze, and we had little concern about Linux. But things came to a grinding halt when we started on the Windows build.

The software is built with MinGW because it uses the Qt framework, and it also has dependencies on the Apache Portable Runtime. These libraries, like many other precompiled Windows development packages such as the MySQL client library, were compiled with Microsoft Visual C++ (MSVC). MSVC and MinGW, we discovered, use different naming conventions for stdcall functions. MSVC exports them as _name@ordinal, but MinGW exports them as name. As a result, the MinGW build failed with “undefined references” link errors when calling stdcall functions exported from the MSVC library.

The MinGW FAQ discusses this problem and offers a solution:

  1. Create a definition file using reimp (for lib files) or pexports (for dll files).
  2. Remove the underscore prefixes from the stdcall functions.
  3. Use dlltool to convert the MSVC library into a MinGW library with the new definition.

That didn’t work. We finally removed the ordinals from the function names, which caused it to compile. But the program wouldn’t run because it couldn’t find the linked functions in the DLL. Finally, after consulting the MSDN documentation for definition files, we changed the build instructions:

  1. Create a definition file using reimp.
  2. For each stdcall function (formatted as _name@ordinal) add a line name = _name@ordinal, allowing MinGW to map its stdcall naming convention to that of MSVC.
  3. Use dlltool to convert the MSVC library into a MinGW library with the new definition.

It worked! To compile the project you must simply:

  1. Download and install the Qt/Windows package, which includes MinGW.
  2. Download reimp and drop it into the MinGW/bin folder.
  3. Download the development packages for the third-party libraries and point an environment variable to that location.
  4. Build the project with the usual qmake/make commands.

We spent a lot of time on the Windows build system, and we’re much wiser now about libraries, export definition files, and the stdcall calling convention. More importantly, this project’s barrier to entry should be low enough to allow any interested developer to contribute.

Additional Reading:

Edit: Updated some URLs.

Blog at WordPress.com.