What’s my Programmer Competency (take 2)

I stumbled across an old blog post of mine, What’s your Programmer Competency?, where I outlined my self-analysis results using Sijin Joseph‘s Programmer Competency Matrix. I figured it was worth revisiting this and see how much things have changed in the eight years since that original blog post.

The Programmer Competency Matrix is measured on a scale from 0 to 3, with 3 being highest.

  • Computer Science: 1.3. Back in 2008, I was a 1. The reasons for the low number today is the same as eight years ago: my computer skills were not learned in the classroom, but in the real world. I’m not a computer scientist (as per the academic definition of computer science). But that’s not a bad thing, because most of the business world doesn’t need computer scientists!
  • Software Engineering: 2.7. Now we’re cooking. The only reason this isn’t a 3 is because I don’t have much experience in automated functional testing and UI testing. This is one area where I’ve continually developed my skills over the years, and the past 8 years have been no exception to that.
  • Programming: 2.9. This is a huge category so I won’t give myself a 3.0, but I’ve got most things in this category nailed. Reusable code? Check. Published frameworks? Check. Ridiculously well-organized code? Absolutely!
  • Experience: 2.3. I’ve got a lot of experience in the technologies that I work with, and I have light experience in a lot of other technologies. Not having experience in things like Erlang and Prolog drags down my score here, but I’m not selling myself as an Erlang or Prolog expert, so I’m happy with my score.
  • Knowledge: 2.4. If my career track was 100% development (and not more of a development/management split) this score would have been higher, but I’ve long ago accepted that part of my value is not just my programming knowledge, but my ability to interact with and communicate with people, not just machines — which is why I’m not spending my entire days sitting behind a computer writing code.

Not bad for a guy who started off as an infrastructure engineer, became a programmer out of necessity and curiosity, and ultimately made a career out of it all. Who would have thought this would happen when I was 12 years old hacking an Atari 800?

What’s your Programmer Competency?

Via Gadgetopia, I stumbled across the Programmer Competency Matrix by IndianGeek.

What is my programmer competency, you ask? Let’s find out…

  • Computer Science: Level 1. This is not surprising to me, as the only formal computer science training I have is an Introduction to Programming class I took in college. I got an “A” in the class and spent half the time telling my classmates that the teacher was wrong. The class taught Pascal, a language I learned to use in 1984, because I was a 14-year old geek who wanted to learn programming beyond BASIC.
  • Software Engineering: Level 2/3. Before becoming a programmer, I was a systems engineer who scripted just about everything you can imagine (which is programming, too, but traditionally not looked at as such). In fact, me and a coworker scripted all Year 2000 compliance testing and updates for 2,500 users in a Fortune 500 company. We used to challenge each other to see who could do more work in one day without leaving our desks. We weren’t lazy; we just preferred to move around during lunch and for the 3:00 half-price cookies in the cafeteria.
  • Programming: Level 2/3. This is a huge category, so it’s hard to say anyone would be a solid “3,” but I feel I’ve mastered a good chunk of the items in here. The one I think I’m best at: “communication.” (Ask anyone who worked with me — peer, subordinate, or manager — and I think they’d concur.)
  • Experience: Level 2. I’m a programmer by evolution, not by initial choice. I’ve spent as much of my career as a programmer as I did as a systems engineer, so I lose points here.
  • Knowledge: Level 2. If I had time to read everything I want to, I’d be better in all categories above.

Continue reading

Bad programming examples (part 1 of x)

There’s no end to the number of bad programming examples we’ve seen in the past or will see in the future. Recently, I saw this one. (This was actual code seen in an actual project.)

try { createDate = Request.Params["createDate"]; }
catch (Exception) { createDate = "-1"; }

Nice and ugly. Aside from a horrible way to implement a try/catch block, it screams of performance issues and unreadable code.

A more proper alternative follows.

createDate = Request.Params["createDate"] ?? "-1";

The same project also had this use of integer parsing.

try { myInt = Int32.Parse(textBox.Text); }
catch (Exception) { myInt = -1; }

In this case, you would use the TryParse method instead:

if (Int32.TryParse(textBox.Text, out myInt))
    myInt = -1;

Exception handling is for exceptions, not for null checking or validations.

Five simple rules for creating delimited text files

Here’s a few tips for those people who provide raw data in text files (think CSV files and the like).

  1. Surround all text fields in single quotes, even if a comma does not exist in a specific instance. By failing to do this, you lose field delimiter consistency on a row-by-row basis, forcing the contents of the field to be parsed separately (i.e. stripping the “optional” quotes).
  2. Use consistent line endings. Pick one and stick with it for all your files. Use either (CR/LF), (LF), or (CR) — and use the same in all your files.
  3. Put column headings in the first row only. This is more a convenience than a necessity. If you make your first row column headings, make sure it is only the first row.
  4. Every row past the first should have an identical schema. Don’t try to be fancy and have different row types in one file. Each file should have the same number and sequence of columns.
  5. Provide delimiters for all columns, even when a row does not have data for them. For example, in a comma-delimited file with five columns, if a row has data in only the first two columns, make sure the row ends with three commas. Failure to do so implies missing or incomplete data.

When text files following these guidelines, I can write a script to import them into a SQL table (using BCP and a simple batch file) in a few minutes. Each guideline that is broken requires additional cleanup steps and more complex data import steps, and adds significant development (and debugging) time that shouldn’t be necessary.

Three tips for grief-free project estimates

Having spent six of the past ten years as a consultant, I’m all to familiar with the practice of estimating. Every client wants an estimate, and every client wants your estimate to be accurate. Of course, clients also don’t want to give you concrete requirements that are needed to give an accurate estimate, either, which compounds the problem.

Scott Hanselman has a nice post about estimating, where he mentions two lessons I learned over the years:

  • Make your estimate, then double it. I actually took this a step further. If an estimate had to be given based on very sketchy requirements, I’d double it twice (effectively quadrupling it). This practice leads to…
  • Under-promise and over-perform. Always make sure your estimate gives you sufficient cushion to come in ahead.

A third lesson he doesn’t mention is to be willing to walk away from a client if your estimate is too high. If a client balks at your estimate (even if you double, or quadruple, it), you can either reduce the scope of your proposed work (and thus reduce the estimate) or walk away. I’ve taken projects that I’ve regretted taking after all was said and done, and most of them can be attributed to me skimping on my estimate because the client was scared off by my original (and, usually, more accurate) estimate.

These lessons go not just for programming projects, but for nearly everything in life. Over-estimate, under-promise, over-perform, and don’t shortchange yourself. Words to live by.

Choosing method names for a cache interface

As part of the overhaul of the WilsonORWrapper, I’m adding a cache service. To do this right, I need to implement an ICacheClient interface. This interface will define the methods and properties which any cache client will need to implement.

Sounds simple on the surface, but I quickly ran into a problem. What do you call your methods? Consider the following possibilities for naming four core methods.

  • Retrieving an item from the cache: Get, Retrieve, Load, Fetch, Read, Select.
  • Placing an item in the cache: Put, Set, Add, Insert, Save, Update.
  • Determining if an item exists in the cache: Contains, Exists, Has.
  • Remove an item from the cache: Remove, Delete, Clear.
  • Clear all items from the cache: Flush, Clear, ClearAll, DeleteAll, RemoveAll.

The worst part of determining a naming convention is that you are generally stuck with it once you choose it, unless you want to implement breaking changes into your application.

Most of the names above were found in some cache API currently in existence today. (I did a Google search to find as many as possible.) Some of them are not recommended because their usage is uncommon (such as Fetch, unless you have a DogCacheClient). I’d also stay away from the *All methods — they sound good on paper, but the difference between Remove and RemoveAll is quite big, and you can call the wrong one if not paying sufficient attention.

With those left over, you can organize them into groups based on similarity of use in well-known applications, languages, and APIs.

  • SQL Style: Select, Insert, Update, Exists, Delete, Clear.
    This style is good if you expect to have separate Insert and Update methods, where Insert can only add new items, and Update is used to change existing items.
  • Collection Style: Get, Add, Contains, Remove, Clear.
    The common naming used in Microsoft and Java collections.
  • Simple Style: Get, Put, Has, Remove, Clear.
    Short and sweet. You can arguably combine Remove and Clear if you’re brave.

Which one should I use for the WilsonORWrapper? One thought is to use the SQL style, since this is a wrapper for an O/R mapper, which is closely tied to SQL. However, I don’t want separate Insert and Update methods. Also, the point of the wrapper is to allow people to write in code, not in SQL, so the relationship to SQL should not dictate style.

That leaves us with the simple style and the collections style. NHibernate uses the simple style in their Cache API. Microsoft’s System.Collections namespace uses the collection style. I’ll go with a combined version: Get, Set, Contains, Remove, Clear, mostly for the reason that users of WilsonORWrapper are probably more familiar with Microsoft Collections than NHibernate, and because I prefer Set over Add.
Have you seen, or used, other naming conventions in your travels?

KISM: Keep it simple, Microsoft

Microsoft, the company who provided the products and tools for millions of people to build careers off (myself included), has often forgotten that the simple solutions are often the best. In a recent blog post, hammett wrote about Microsoft’s missteps in this area and their focus on YAGNI (You Ain’t Gonna Need It) — at least, where “You” refers to most people.

[Digression: Someone at some point commented on Microsoft Word that “90% of the features are used by 10% of the people”. If I was designing a product and 10% of my features were used by 90% of the people, and the other 90% of the features were used by 10%, I’d either write two products, or I’d write one product that was incredibly extensible using a plug-in architecture.]

This feature-bloat approach to technology reminds me of Microsoft’s Enterprise Library and the Data Access Application Block (DAAB). In the first release of the DAAB, you can call a parameterized stored procedure and get a DataReader back using one line of code, as illustrated below.

IDataReader reader = SqlHelper.ExecuteReader(
    connectionString, CommandType.StoredProcedure, storedProc, 
    new SqlParameter("@ID", 1));

In the latest DAAB (part of Enterprise Library 2.0), this becomes:

Database db = DatabaseFactory.CreateDatabase();
DbCommand cmd = db.GetStoredProcCommand(storedProc);
db.AddInParameter(cmd, "ID", DbType.Int32, 1);
IDataReader reader = db.ExecuteReader(cmd);

I also should mention I also should mention that the latter example also requires a special configuration section added to your application config file, whereas the former just requires a connection string (which likely you already include somewhere in your application configuration).

The first example hides the complexity that you may not need in most circumstances. Granted, you can still use (or write your own) SqlHelper, but why break away from this entirely?

Simplicity is a wonderful thing. Let’s hope Microsoft finds it again. Give power to those who need it, and simple elegance to those who don’t. It’s not hard to do both, if you accept the fact that one size does not fit all.

Copying an ADO RecordSet in Visual Basic

The ADO RecordSet object’s Clone method does a great job of making a duplicate copy of the RecordSet, with one major caveat: any changes to the clone are duplicated on the original. It’s more like a shallow copy than a deep copy.

To make an actual copy of a disconnected ADO RecordSet in Visual Basic, use a method like the one shown below, which was largely taken from Francesco Balena’s article on devx.com:

Private Function CopyRecordset(rsSource As ADODB.Recordset) As ADODB.Recordset
    Dim rs As ADODB.Recordset
    Dim pb As New PropertyBag
    ' create a copy of the recordset
    pb.WriteProperty "rs", rsSource
    Set rs = pb.ReadProperty("rs")
    ' release the memory
    Set pb = Nothing
    Set CopyRecordset = rs
End Function

How does ProxyBuster.net work?

I received an email today from Johnny B. (Good?), in which I was asked a question about a Web service I provide, ProxyBuster.net. The email was:

Hello. I had a little question and I’ll be so glad if you can answer me. I wanted to ask: What system or platform does ProxyBuster use in its process, is it CGI, PHP, ASP, JavaScript, or something else? I mean which one of these can manage to trick the firewall and access a forbidden file?

My response is: None of those, actually. The “trick” is understanding how firewalls work. Most firewalls/proxies do one of a few things:

  • Block by IP address.
  • Block by domain name.
  • Block by URL text.
  • Block by file extension.
  • Block by HTTP content type.
  • Block by actual content type.

Let’s say you want to access Google’s home page (http://www.google.com/index.html). A firewall can:

  • Block the IP address 216.239.39.99.
  • Block the domain name, www.google.com.
  • Block a URL that has “www.google.com” in it (as in http://www.google.com).
  • Block all files ending in “.html”.
  • Block all files that have the HTTP header content-type “text/html”.
  • Block all files that are actually text files.

In this case the first three bullet items are effective, but the last three would limit your ability to use the Internet as a whole. Usually binary files (ZIP files, EXE files, etc) are restricted by the latter three types of blocks, and entire sites by the first three types of blocks.

So how does ProxyBuster do its magic?

First, if your firewall/proxy blocks the IP address, that’s not a problem to ProxyBuster, because you do not connect to the site directly. ProxyBuster connects to the site and reads your data, then provides it to you. Of course, if your firewall/proxy blocked www.proxybuster.net or our IP address, you’d be out of luck using our service because you wouldn’t be able to connect to it!

Next, ProxyBuster returns the file to you in one of a few formats. First, the file is sent to you as “download.aspx” — which is a very generic file name and file extension that would be too restrictive to block. (For example, blocking “.aspx” files would make it impossible to browse Microsoft’s Web site.) Next, we give you the option to receive the file with an HTTP content-Type header “text/plain” even if the file is not a plain text file. Last, if your firewall/proxy blocks binary files, we can send you the file in a text-encoded format (so it’s transmitted as a text file); all you need to do is decode it after you receive it (which does require a separate utility).

I hope this sheds some light to the inner workings of ProxyBuster!