Archive for the ‘testing’ Category

Cucumber is Cool and so is The Cucumber Book

In Book Reviews, Computer science, Software Engineering, testing on December 1, 2011 at 2:37 pm

So, I’ve been learning Cucumber—a tool for writing executable acceptance tests—in preparation for my upcoming course, Testing for Developers. The course will be heavy on the theory of testing, but from the viewpoint of the developer. How do you build testability into your code, what techniques can you use, and what tools are there that will help you do test automation. Most of my students will already know the basic xUnit tools, but we will be going much more into mock objects and other techniques for writing really good unit tests. We’ll hit Test-Driven Development (TDD) hard and we’ll look at how TDD and Behavior-Driven Development (BDD) work together, etc. We’ll also be wanting to work with embedded software systems and building test tools when necessary, or at least the glue to help tools work together.

This is an ambitious undertaking. I had heard about Cucumber when I looked at RSpec several months ago. RSpec was a bit too limited for what I wanted. Then I got a copy of The Cucumber Book by Matt Wynne and Aslak Hellesøy, published by The Pragmatic Bookshelf. After reading most of the book and a few weeks of playing around with Cucumber, I’m hooked. Cucumber takes the place of several tools I was thinking about introducing to my class, and allows me to into depth in several areas, using a single platform. If you’re thinking about learning Cucumber, or want to learn something about it, get this book.

The style of the book is great. It’s easy to read and describes how to use Cucumber in several bite-sized pieces. Some readers  may even find that the pieces are nibble-sized and want bigger bites. I urge you to avoid this. Install the software and go through each exercise in the steps described, even though you know what you might need to do in the next few steps. This will help you develop your intellectual muscle memory for using Cucumber. Take time to taste each bite, no matter how small it may be, because there is some new spice that was added that might be subtle if you skip over the step.

The first part of the book, chapters 1-6, introduces you to Cucumber fundamentals. Actually, this part does a little more. It teaches you something about BDD as well. If you’re really not interested in learning about BDD, you can skim chapters five and six, but I would not recommend skipping them. By the end of the first section I was able to do some useful things with Cucumber and used it to demonstrate requirement specification to my current undergraduate software engineering class.

The second part of the book, chapters 7-10, are chapters you need to read and work through if you want to become competent with Cucumber. Again, some of the discussions seemed too small to me, but after reflecting upon what I learned, I think the level is just right. Some of what you learn in this part of the book is how to work with some other tools to use databases and Web pages in your testing.

The third part of the book is one that does not have to be read in any special order, or even at all. But, if you really want to understand how to use Cucumber to define and (acceptance) test Web applications (written in Rails or otherwise) and other types of applications, you need to have these chapters available. So, skim them at least and then refer to them as necessary.

Is there a downside to this book? Well, not to the book itself. The fact is that Cucumber, like many other open source projects works in a specific inter-dependent ecosystem. Setting up Cucumber and its dependents and other pieces you might want like Sinatra, Capybara, etc. can be frustrating; especially if you want to do this on multiple platforms. If you’re not a Ruby expert, you may run into problems and will need to do some searching for answers or post to the forum for the book at the Pragmatic Bookshelf. But don’t let this scare you away. Cucumber is a good tool to learn. Whether you use it regularly or only for special occasions or specific projects, you will think a little bit differently about how you build software.


Arduino Part 2

In embedded software, testing on January 31, 2011 at 2:22 am

Processing Not C

Upon first look, I thought that the Arduino IDE supported C and C++. It actually looks like it. But after reading the first couple of chapters of Getting Started With Arduino, I learned that it actually supports a variant of the Processing language. This is nice for getting started quickly; unfortunately, it does not necessarily allow the incorporation of testing frameworks like Unity and CppUnit. So my next task is to get Eclipse configured for the Arduino. The starting point for this exploration is in the Arduino playground pages on Eclipse. It seems like the instructions are quite straight forward.

The way the Arduino IDE works is that it translates the source to C and then passes it to gcc. I may be able to integrate one of the unit testing harnesses with this, but not for the initial class since the first group is a set of developers who are programming in C and C++. One of the requirements of the class is that we use these languages as much as possible. I will, however, try to introduce domain specific languages for testing as part of the course content.

Starting with Eclipse

I might as well jump right it. I’ve already got Eclipse set up for the AVR processors on my Mac. This required first installing the CrossPack tool chain. This installs the AVR gcc compiler and other AVR development tools. There are other tools required for Windows and Linux. These are identified in the Arduino playground.

After a couple of hours of reading and hacking, I have a blink program written using Eclipse and running on the Arduino. Here’s what I ended up doing. Most of the information comes from the Arduino playground – Eclipse page already mentioned. The specific things I did were:

  1. Compiled my own static library for the diecimilia board. I called this libDiecimiliaCore.a. I’ll have to compile another one for the Uno.
  2. Created my first project for the Arduino. It references the header files in the library project and includes the static library when linking.
  3. Copied the main.cxx to the project and renamed it main.c since this is simple C code.
  4. Wrote the “blink” application from the Arduino IDE. I simply copied it. Now, the interesting thing is that the delay is over an order of magnitude faster than with the Arduino IDE. I think one of the tasks for the class will be to calibrate the delay loop. This is a good test to start with.

The instructions for doing this are quite clear. It is fairly tedious and time consuming to get this workiong the first time. Next up is making a template / configuration so I don’t have to go through all of the settings every time. At least I’m seeing some progress.

Arduino Part 1

In embedded software, testing on January 29, 2011 at 2:25 am

So, I received my Arduino Starter Kit today and want to get started understanding it so I can build enough examples and exercises for the Testing Embedded Software for Developers class. The Arduino is much more low level than the Neuron Robotics DyIO, but it’s much less expensive and there’s more documentation available right now. Of course I’ve got the folks from Neuron Robotics available on campus and nearby to help me figure out what’s happening. But, I think it’s about time for me to dive in and figure out some low level concepts. Later on, possibly in future offerings, I can add the DyIO to the course.

You have to realize that I’m very hardware-phobic and really get nervous when I have to touch hardware. It stems from an experience I had building an 8-bit computer in the late 1970s. I reversed polarity on the board and all of the diodes exploded. It cost more to get that fixed than most PCs cost today. Ever since, I’ve always feared damaging components beyond repair. Luckily the Arduino isn’t that expensive and I don’t think it’s going to surge back to my laptop.

I had an Arduino Diecimila that someone gave me to play with. The starter kit came with the Arduino UNO so I’ve at least got two systems in case I blow one out. This could be interesting.

Anyway, I’ve loaded performed the Getting Started with Arduino on MacOSX. It went really smoothly. It took about 15 minutes from starting to download the Arduino IDE to getting the blinking LED. Nice and easy and I didn’t break anything. That certainly builds confidence. I do, however, want to use the Eclipse environment with the AVR plug-in for the class. I think it will make some of the testing easier, but I’m not sure about that yet.

Now I’m going to start reading some of the tutorials and play around with the board. It looks like you build libraries for use with the Arduino and IDE in C++. This is cool. But it also means that there’s an awful lot to learn in a short time. I’m not sure which things are critical. If anyone has any ideas on what would be the most useful, please let me know. Post a comment here.

Getting ready for testing embedded software for developers

In embedded software, Teaching, testing on January 26, 2011 at 8:57 pm

After a couple of months of reading about embedded systems on-an-off and how to go about testing them, I’m finally done procrastinating and putting together the course (with a whole two months to spare). This should be fun, and a good course that we can build upon. This particular one is specifically for corporate education at WPI. There are companies out there who are really eager to get their developers trained in testing techniques for the systems they’re building. I think it will be fun, and it will also help me improve the standard testing course that I’ve offered a couple of times.

I’ve chosen a couple of good books for this course, which is a graduate-level course. The first is Introduction to Software Testing by Ammann and Offutt. This is a really nice book that covers testing with just enough theory and a lot of practical advice. I think it will appeal to developers quite nicely.

Cover of the text book


The second book is Test Driven Development for Embedded C by Jim Grenning. This is the book that I’m learning a lot about embedded systems development. Combining this with the previous book should make for a great course. Of course, I need to go back and remember some of the dirty little corners of C and C++, but it should be fun.

Cover of Test Driven Development for Embedded C


The course will not be simply learning the theory and practice, but putting the knowledge to work by working on projects. Projects make things interesting, but somewhat difficult for a course like this. The students will be practicing software engineers who are already working on embedded code so the challenge will be to try and simulate enough of a real system without the complexity and size of industrial systems. I’m not sure how this will work, but I think it’s possible to make interesting and reasonably challenging problems.

I’ve decided for the first offering to use the Arduino board for the main development hardware. I’d like to incorporate different things like Sun SPOTs and the Neuron Robotics DyIO, but not right now. Maybe these will go into a future offering.


The hardware

The other challenge is how I’m going to talk and keep a class interested for four hours at a clip. I’d rather two days a week, but that’s not possible right now. The last time I did a four hour course was one summer teaching foundations of computer science. I bored myself to death.