As I went down the lines counting, I could not believe the amount of stress that I induced in those being counted.  People quickly began to worry about whether they were too far back in line to get a phone, or even if they were in the right line.  I was stopped about every 25 people asking what their number in line was. I had to explain over and over that I wasn’t counting for any purpose.

For the record, by the time 7 AM rolled around there were 700 total people in the two lines.  During that time I’m sure the line grew, because I started with the no-reservation line first, and then proceeded to the reserved line, which wound around the building in a different direction.  Between 7AM and 7:20, another 82 people showed up in the reserved line behind me.

Over the past 2 years I’ve worked on a small bit of glue to make selects, updates, inserts and deletes work easily from Objective-C.  This includes support for converting MySQL types to Cocoa types, working correctly with garbage collection, and support for reading and writing large blobs.

It is possible to do all these things with the generic MySQL C connector, and if your needs are simple that can be a quick way to access a MySQL databases, since all C libraries can easily be called from Objective-C.  However, once you start dealing with more advanced topics, the C connector can become burdensome.

If you are working on intel platforms only, you should grab the intel only version.  This supports both 32 and 64 bit intel platforms, using version 6.0.2 of the MySQL C Connector.

ObjC_mysql_intel.zip 5.5 MB – 32/64 bit

I still have some client programs running on PPC boxes, but MySQL has dropped support for PPC based OSX in the most recent versions of the mysql_connector.  So I have an older version that I occasionally update, although not as often as the intel only version.  This version uses an older version (5.0.51a) of the MySQL C connector, and supports both 32 bit PPC and Intel platforms.

ObjC_mysql_ppc.zip 2.3 MB – 32 bit intel and PPC.

Both versions use an identical syntax and are for the most part interchangeable.

Both require that garbage collection be enabled in your application.  This means this version won’t support versions of iPhone OS that don’t include garbage collection. Which is pretty much all of them, as of the writing of this post.

Both include the headers and a static library for the MySQL C connector.

While I have used these libraries to process well over half a billion rows of MySQL data, they are not by any means guaranteed to be free of bugs.  The most recent fix was a leak in MysqlUpdate.  If you are interested in getting updates, subscribe to the MySQL Category Feed, or email me for access to the subversion repository for the project.

Adding to Your Project

Once you have extracted the zip, add everything inside to your project.  The static mysql library will be linked into your application, and the headers and all the necessary bits to compile should just fall into place.

Forming the connection

You connect by creating an instance of MysqlConnection.  The best method for starting is

+ (MysqlConnection *)connectToHost:(NSString *)host
                              user:(NSString *)user
                          password:(NSString *)password
                            schema:(NSString *)schema
                             flags:(unsigned long)flags;

The best option for flags is MYSQL_DEFAULT_CONNECTION_FLAGS

If you are connecting to a MySQL server that will kick you if inactive, call the method

-[MysqlConnection startIdle]

to start a low impact periodic select that will run once every 60 seconds. Four additional methods add support for transactions

Performing Selects

Selects are pretty straight forward. Use MysqlFetch, and pass a SQL select command. You can then look at the results property to get an array of dictionaries. Each dictionary represents a single row. They key will be the name of each column, and the value will be the Objective-C type that matches the column. If you aren’t sure of the names of the fields, you can look at the fields an fieldNames properties for more details.

Example:
MysqlConnection *connection = [MysqlConnection connectToHost:@"serverName"
                                                        user:@"aUserName"
                                                    password:@"aPassword"
                                                      schema:@"myForumName"
                                                       flags:MYSQL_DEFAULT_CONNECTION_FLAGS];
MysqlFetch *userFetch = [MysqlFetch fetchWithCommand:@"select U_Number,U_Username from w3t_Users"
                                        onConnection:connection];
QLog(@"There are %d members",[userFetch.results count]);
for (NSDictionary *userRow in userFetch.results) {
  NSNumber *userNumber = [userRow objectForKey:@"U_Number"];
  NSString *userName = [userRow objectForKey:U_Username"];
  NSLog("%@ %@",userNumber,userName);
}

Performing Deletes

Deletes, inserts, and updates are very similar. To perform a delete:

MysqlDelete *deleteCommand = [MysqlDelete deleteWithConnection:connection];
delete.tableName=@"w3t_Users";
delete.qualifier=[NSDictionary dictionaryWithObject:@"3" andKey:@"U_Number"];
[deleteCommand execute];

You can then look at the affectedRows property to find out how many rows were deleted.

Performing Inserts

Inserts allow you to insert NSString, NSNumber, NSData, and NSNull. All other classes are converted using their -description method. The insert code uses bindings instead of composing a SQL string, so you don’t need to worry about escaping special characters, or filtering user input.

MysqlInsert *insertCommand = [MysqlInsert insertWithConnection:connection];
insertCommand.table =@"w3t_Users";
NSData *picture = [NSData dataWithContentsOfFile:@"sample.png"];
NSString *password=@";'!#";
insertCommand.rowData=[NSDictionary dictionaryWithObjectsAndKeys:@"Karl Kraft",@"U_Username",
                                                          picture,@"U_UserImage"
                                                          password,@"UserPassword"
                                                          nil];
[insertCommand execute];

After the insert, you can get the auto increment rowid by using the rowid property on your MysqlInsert instance.

Performing Updates

Once you have mastered delete and insert, updates are a simple cross of the two. Specify a qualifier, like you would for deletion, but rowData like you would use for an insert. Only the specified fields in the rowData are modified. Again you can pass NSString, NSNumber, NSData, and NSNull, and the values are inserted using bindings so that you don’t need to provide odd escapes.

MysqlUpdate *updateCommand = [MysqlInsert insertWithConnection:connection];
updateCommand.table =@"w3t_Users";
updateCommand.qualifier=[NSDictionary dictionaryWithObject:@"3" andKey:@"U_Number"];
NSData *picture = [NSData dataWithContentsOfFile:@"sample.png"];
NSString *password=@";'!#";
updateCommand.rowData=[NSDictionary dictionaryWithObjectsAndKeys:@"Karl Kraft",@"U_Username",
                                                          picture,@"U_UserImage"
                                                          password,@"UserPassword"
                                                          nil];
[updateCommand execute];

Like delete you can query the affectedRows property to find out how many records were updated.

Catching exceptions

Any exceptions while performing the mysql commands are thrown as MysqlException or a subclass of MysqlException. Common reasons for exceptions being throw are:

• Unable to connect to the database, or wrong username / password
• Commit or rollback fails
• Unsupported field type in a MysqlFetch
• Unqualified delete or update commands
• Any underlying mysql error

MYSQL_LOGGING

As a final useful trick you can define MYSQL_LOGGING in the CFLAGS of your project, or in a common header file, and useful debugging information will be generated.

Either I’m missing something, the initial iPad apps are going to suck, or we haven’t yet been told that iPad-native apps won’t be available for some period of time after the iPad’s launch.

You are missing something.

The problem, of course, is that before day one, we won’t have iPads ourselves for development and testing. This wasn’t a problem for iPhone development: by the time the SDK was released, we had all been using iPhones for many months. We knew how iPhone apps should look and behave, and we could test our apps on our iPhones during development for three months before anyone could sell apps to customers.

No, we didn’t. You might have, but thousands of others were left out in the cold, unwelcome to develop for the iPhone. Apple tightly controlled who they allowed in, and many didn’t get developer keys until months after the App Store was opened for business. Yes, we could develop on the simulator, but not on the phone itself. This preferential treatment continued with the release of later beta SDKs and Firmwares. The entire time this was going on Apple was very coy about what they were doing.

You have based your 3 possibilities on a bed of false assumptions.

You have assumed that you can go to the local Apple Retail store, buy an iPad and install software.

This is not supported in the current SDK. At this point you can’t even compile code to the A4.

Apple hasn’t said when it will be supported.

Most importantly there is no guarantee that your current developer certificate will allow you to cryptographically sign and install software on the iPad at all.

Your certificate may work for iPhone and iPod touch only.

Apple may require a separate enrollment fee to get a iPad certificate.

Apple may limit iPad certificates to a select few developers. This limitation may be temporary or permanent.

You have assumed that Apple will accept iPad apps from you sometime on or before “day 1″.

Apple has made no such commitment.

They have committed to their future iPad users that iPad specific apps will be ready on day 1, but they haven’t committed to any developers (AFAIK) that they can submit them.

When it comes to the AppStore, Apple is not a matchmaker helping developers connect to users in a frictionless manner. It sure seems that way and as an indie developer I like the AppStore, but it is what it is, which is Apples playground.

You have assumed that there is a submission queue.

Developers like data structures like FIFO, and lots of them refer to the “submission queue” but I have never seen documentation from Apple that submissions are handled in a first come first served manner. Anecdotal evidence seems to indicate that the are several submission piles, and they are serviced at different rates.

I have kept all the weekly sales reports going back to when my applications were first available. So I was hoping I would finally be able to see my grand total for each app. The only problem is that the grand total is based on daily reports, not weekly reports.
So I created a small command line tool called SplitWeek that would break each week file into 7 daily files.

Each sale is assigned randomly to one of the seven days in the weekly period, so there are no rounding errors.

Operation is simple. Just run the command from Terminal, passing the path to the weekly file as the first argument:

./build/Debug/SplitWeek reports/S_W_20080825.txt

The 7 files will then be generated and saved to the current working directory.

2009-10-29 11:05:36.933 SplitWeek[53735:903] S_D_20080825.txt
2009-10-29 11:05:36.934 SplitWeek[53735:903] S_D_20080826.txt
2009-10-29 11:05:36.935 SplitWeek[53735:903] S_D_20080827.txt
2009-10-29 11:05:36.935 SplitWeek[53735:903] S_D_20080828.txt
2009-10-29 11:05:36.936 SplitWeek[53735:903] S_D_20080829.txt
2009-10-29 11:05:36.937 SplitWeek[53735:903] S_D_20080830.txt
2009-10-29 11:05:36.937 SplitWeek[53735:903] S_D_20080831.txt

Once you have created all your daily files, zip them up, and upload them to MyAppSales using the web server built in to the application.

And speaking of the built in web server, here is a quick tip on how to connect to it. When you enable the server, it will show the IP and port you need to enter into Safari to connect. In the screen shot here, I’m using the simulator, so it doesn’t even give an IP.

Instead of entering the IP and port, just use the Safari Bonjour support, and you will find it listed under Webpages.

The original post by Robert Scott complained that this code wasn’t showing Leaks in instruments even though it should be leaking a NSString with every pass.

Robert wrote:
So I guess there must be
something fundamental I don’t understand about leaks because I don’t
see why your examples cause leaks and this code does not:

  static int k;
  NSString *leakyString = [[NSString alloc]  initWithFormat:@"%1d", ++k] ;
  myLabel.text = leakyString;

The answer:

Because they aren’t leaks. If you try to use NSString to learn about
Objective-C memory management you will get burned over and over again,
because it uses tons of behind the scene tricks. NSString is not a
good place to learn to leak detection.

You are not the only one with references to those objects. This is
going to be long and involved, so get a cup of your favorite beverage,
download the example code and cuddle up with Xcode:

First off make sure you are using Xcode 3.1.2 for this demo, with the simulator set for 2.2.1. Effects may vary by platform and by version.

- (IBAction)pushMe:(id)sender;
{
  static int k;
  NSString *leakyString = [[NSString alloc]  initWithFormat:@"%1d", ++k] ;
//	myLabel.text = leakyString;
  NSLog(@"%@ %p %p",leakyString,leakyString,myLabel.text);
}

This code adds the printing of the pointers to leakyString and
myLabel.text For now the assignment to myLabel.text is commented
out. Run the code and watch the output of the debugger console. You
should get something like this:

2009-04-22 22:17:11.443 LeakSampler[27569:20b] 1 0xa03ca9f0 0x508540
2009-04-22 22:17:11.583 LeakSampler[27569:20b] 2 0xa03c3640 0x508540
2009-04-22 22:17:11.732 LeakSampler[27569:20b] 3 0xa03c9820 0x508540
2009-04-22 22:17:11.873 LeakSampler[27569:20b] 4 0xa03c8e20 0x508540
2009-04-22 22:17:12.013 LeakSampler[27569:20b] 5 0xa03c9dc0 0x508540
2009-04-22 22:17:12.153 LeakSampler[27569:20b] 6 0xa03ca520 0x508540
2009-04-22 22:17:12.294 LeakSampler[27569:20b] 7 0xa03c92a0 0x508540

Keep pushing until you get to:

2009-04-22 22:17:29.193 LeakSampler[27569:20b] 48 0xa03c9cd0 0x508540
2009-04-22 22:17:29.357 LeakSampler[27569:20b] 49 0x53e040 0x508540
2009-04-22 22:17:29.505 LeakSampler[27569:20b] 50 0xa03c7100 0x508540
2009-04-22 22:17:29.662 LeakSampler[27569:20b] 51 0xa03c6120 0x508540
2009-04-22 22:17:29.810 LeakSampler[27569:20b] 52 0xa03c4c30 0x508540
2009-04-22 22:17:29.958 LeakSampler[27569:20b] 53 0x5145d0 0x508540
2009-04-22 22:17:30.107 LeakSampler[27569:20b] 54 0xa03c99a0 0x508540

Note that for run 49 and 53 the results were different. They are in a
much lower section of memory.

Now use “Start with Performance Tool -> Leaks” and run the app.
Repeat all the way up to 54, (You will have to use Console.app to
watch the progress)

You should notice a few things. The pointers for 1..54, except for
for 49 and 53 were the EXACT same on this run. The only items that
show up as leaks are the entries for 49 and 53, which have the lower
memory addresses.

4/22/09 10:19:35 PM LeakSampler[27582] 1 0xa03ca9f0 0x508570
4/22/09 10:19:36 PM LeakSampler[27582] 2 0xa03c3640 0x508570
4/22/09 10:19:36 PM LeakSampler[27582] 3 0xa03c9820 0x508570
4/22/09 10:19:36 PM LeakSampler[27582] 4 0xa03c8e20 0x508570
4/22/09 10:19:36 PM LeakSampler[27582] 5 0xa03c9dc0 0x508570

You might be tempted at this point to think Leaks doesn’t detect leaks
with high memory addresses. You would be wrong. Play some more.

Remove the comment for setting the label, and start again without
Instruments.

- (IBAction)pushMe:(id)sender;
{
  static int k;
  NSString *leakyString = [[NSString alloc]  initWithFormat:@"%1d", ++k] ;
  myLabel.text = leakyString;
  NSLog(@"%@ %p %p",leakyString,leakyString,myLabel.text);
}

Note the first 5 lines of output.

2009-04-22 22:24:53.204 LeakSampler[27717:20b] 1 0xa03ca9f0 0xa03ca9f0
2009-04-22 22:24:53.939 LeakSampler[27717:20b] 2 0xa03c3640 0xa03c3640
2009-04-22 22:24:54.431 LeakSampler[27717:20b] 3 0xa03c9820 0xa03c9820
2009-04-22 22:24:54.790 LeakSampler[27717:20b] 4 0xa03c8e20 0xa03c8e20
2009-04-22 22:24:55.087 LeakSampler[27717:20b] 5 0xa03c9dc0 0xa03c9dc0

myLabel.text should be getting a copy of the string:

@property(nonatomic,copy)   NSString *text; //  default is nil

Yet the pointer for both leakyString and myLabel.text are exactly the
same. This is because NSString is non-mutable and copy of a non-
mutable object can simply be replaced with a reference to the exact
same object to reduce memory consumption.

Now a few more tests. Instead of NSStrings, create NSMutableStrings

2009-04-22 22:27:06.118 LeakSampler[27739:20b] 1 0x50c450 0xa03ca9f0
2009-04-22 22:27:06.431 LeakSampler[27739:20b] 2 0x53e7b0 0xa03c3640
2009-04-22 22:27:06.508 LeakSampler[27739:20b] 3 0x518200 0xa03c9820
2009-04-22 22:27:06.898 LeakSampler[27739:20b] 4 0x526210 0xa03c8e20
2009-04-22 22:27:07.267 LeakSampler[27739:20b] 5 0x53ef00 0xa03c9dc0

Note that the mutable strings are in low memory, yet the copies have
the same addresses as the very first run. The addresses for the
mutable strings will be different on each run as various other parts
of the system allocate and release memory.

Now for the fun part. Set a break point on the line that reads:

  static int k;

And fire up the program in GDB. Press the button one time and at the breakpoint, before your call to initWithFormat:, print the objects at the first five memory addresses that keep showing
up:

(gdb) po 0xa03ca9f0
1
(gdb) po 0xa03c3640
2
(gdb) po 0xa03c9820
3
(gdb) po 0xa03c8e20
4
(gdb) po 0xa03c9dc0
5

The NSString objects that you thought were all yours, that you loving
created with a clever call to initWithFormat:, turned out to just be
filthy dirty replicants of NSStrings generated before your code even
ran.

For fun, you can find some more strings that are already created:

(gdb) x/20x 0xa03c3640
0xa03c3640:	0xa03b84a0	0x000007c8	0x93c78a10	0x00000001
0xa03c3650:	0xa03b84a0	0x000007c8	0x93c80194	0x00000014
0xa03c3660:	0xa03b84a0	0x000007c8	0x93c801ac	0x00000002
0xa03c3670:	0xa03b84a0	0x000007c8	0x93c801b0	0x00000011
0xa03c3680:	0xa03b84a0	0x000007c8	0x93c801c4	0x00000001

Looks like tightly packed objects in a lookup table:

(gdb) po 0xa03b84a0
NSCFString

Yup, a bunch of NSCFString objects. A quick examination shows this
portion of the table to be:

(gdb) po 0xa03c3650
HelveticaNeue-Italic
(gdb) po 0xa03c3660
88
(gdb) po 0xa03c3670
orange_middle.png
(gdb) po 0xa03c3680
B

If you add this line of code what do you think you get?

NSLog(@"%p",[[NSString alloc] initWithFormat:@"%@",@"orange_middle.png"]);
2009-04-22 22:38:05.504 LeakSampler[27848:20b] 0xa03c3670

In Summary, I Repeat: If you try to use NSString or other non-mutable objects to learn about
Objective-C memory management you will get burned over and over again,
because it uses tons of behind the scene tricks. NSString is not a
good place to learn to leak detection.

As for why 49 and 53 aren’t in the table? Well I think that is blindingly obvious once you think about it.

I’m not a big fan of debugging from log statements, but they can occasionally be useful.

What follows is details on building a full NSLog() replacement that turns off with a single #ifdef in the header file that contains the function. You can either download the m and h file, or continue on to the extended entry for an explanation of how it works and ways to make it more useful for your particular needs.

Start with a basic Xcode Foundation Tool. This will contain a main.m file that uses NSLog to print “Hello World!”

  NSLog(@"Hello, World!");

The output will look like this:

  2009-03-23 10:57:13.619 DebugLog[11941:807] Hello, World!

Add a DebugLog.h and DebugLog.m to the file. DebugLog.h should declare a method DebugLog() that will be the replacement for NSLog()

  void DebugLog(NSString *format,...);

And DebugLog.m should contain the implementation of that function. A particular item to note is that NSLog only adds a newline to the end of the NSLog format if one is not already there. This function duplicates this feature of NSLog()

  void DebugLog(NSString *format,...) {
    va_list ap;
    va_start (ap, format);
    if (![format hasSuffix: @"\n"]) {
      format = [format stringByAppendingString: @"\n"];
    }
    NSString *body =  [[NSString alloc] initWithFormat: format arguments: ap];
    va_end (ap);
    fprintf(stderr,"%s",[body UTF8String]);
    [body release];
  }

You can download the project at this stage. If you play with the project at this point you will notice the output is different than NSLog().

  // With NSLog
  2009-03-23 11:01:32.936 DebugLog[12085:807] Hello, World!
  2009-03-23 11:01:32.938 DebugLog[12085:807] Hello, World!
  // With DebugLog
  Hello, World!
  Hello, World!

NSLog() includes the time, process name, process id and thread id as a prefix on every message. I find this excessive in most cases, but you can add it in if you wish. More on this later.

In the next version of the project, disable the function except for Debug builds. Do this by editing DebugLog.h to have an #ifdef that turns DebugLog() into a macro, and rename the function from DebugLog to _DebugLog. By using a #else we can disable DebugLog for any non DEBUG builds.

  #ifdef DEBUG
  #define DebugLog(args...) _DebugLog(args);
  #else
  #define DebugLog(x...)
  #endif
  void _DebugLog(NSString *format,...);

Rename the function in DebugLog.m as well.

  void _DebugLog(NSString *format,...) {

Finally, setup the build settings to include a DEBUG flag. For more deatil or for iPhone configurations, see the the post that inspired this one.

Now try building both the Debug and Release version. Run from the command line, only the Debug version will produce the debug lines

  explorer:karl/tmp/DebugLog%./build/Debug/DebugLog
  Hello, World!
  explorer:karl/tmp/DebugLog%./build/Release/DebugLog
  explorer:karl/tmp/DebugLog%

Again, you can download the project at this stage.

Now take the function from better to best. By using some preprocessor directives we can embed the name of the file and line number where the DebugLog() is called, or have it print a function or method name.

Change the macro definition and function declaration:

  #ifdef DEBUG
  #define DebugLog(args...) _DebugLog(__FILE__,__LINE__,args);
  #else
  #define DebugLog(x...)
  #endif
  void _DebugLog(const char *file, int lineNumber, NSString *format,...);

and change the fprintf() in the implementation to print the passed arguments:

  fprintf(stderr,"%s:%d %s",file,lineNumber,[body UTF8String]);

The output will now look like this:

  /Users/karl/tmp/DebugLog/main.m:15 Hello, World!

The __FILE__ is expanded to the complete path to the file, which is usually more verbose that I need. It can be trimmed down to the actual file name by using an NSString method:

  ...
  NSString *fileName=[[NSString stringWithUTF8String:file] lastPathComponent];
  fprintf(stderr,"%s:%d %s",[fileName UTF8String],lineNumber,[body UTF8String]);
  ...

This will produce this output:

  main.m 15 Hello, World!

Again, you can download the project at this stage.

Finally, here is a more tricked out version that prints the function or method name instead of the file name, and prints the name of the thread.

  explorer:karl/tmp/DebugLog%./build/Debug/DebugLog
  MainThread/main (main.m:16) Hello, World!
  MainThread/+[SampleClass debugExample] (SampleClass.m:16) A sample debug message
  MainThread/-[SampleClass debugExample] (SampleClass.m:21) Another sample debug message

The final project can be downloaded here. Stick the #import into your prefix header, and you can liberally sprinkle DebugLog commands all over your code.

This has been driving me crazy for awhile now, so I looked into the problem and found a “solution”. While I can’t vouch that it will work for everyone, I wanted to share it in the hopes that it might work for some.

This is going to be long so that everyone can understand the cause and possible soutions.

The problem firsrt showed up when I started using SVN 1.5. I upgraded all my repositories on my linux and mac servers, and the mac server was working well, but the linux server would ocassionally just hang in Versions. As part of the 1.5 upgrade there are some caches that you can either prebuild or build as needed, so I assumed the slowness was just those caches slowly being updated.

That was until this morning when I had the problem with a brand new empty repository. I found that when Versions was running, access to the repository was slow even from the command line. But if I wasn’t running Versions, performance was lightning fast from the command line.

It turns our the problem is related to the way SVN 1.5 uses random numbers. On linux (and OS X), there are two sources of random numbers. One is /dev/random, and the other is /dev/urandom. /dev/random produces true cryptographically secure random numbers. If there isn’t enough entropy available it will hang waiting for secure random numbers to become available. urandom on the other hand will sacrafice cyrptographica quality and always produce random numbers.

Svnserve on my linux box uses /dev/random, and if the random numbers aren’t available it blocks until they are. This in turn leads to Versions blocking waiting for svnserve to respond, which will happen eventually as sources of entropy like network traffic occur.

The problem is compounded when you turn on “Show Updates from Repository”. For me this is the entire reason for having a program like Versions, so that I can keep track of what repositories I’m working with have experienced changes, so I can make sure my source stays up to date and that I’m aware of who is making what changes to the source.

The problem is that the SVN server doesn’t do much besides serve svn connections. So there isn’t much entropy available. Eventually /dev/random runs out of random numbers and svnserve will hang, causing Versions to hang. Versions should be a bit smarter about realizing that if a request isn’t critical (like say a commit or update) that it can just terminate the connection on quit.

The solution is to either rebuild the Apache Portable Runtime library that svnserve from source and change it to use /dev/random instead of /dev/urandom. This would need to be done on the server.

The other choice is to simply remove /dev/random and replace it with the equivalent of urandom. This will affect all programs running on the server.

The commands are not for the faint of heart and can easily render the box unusable if done improperly. These are for a Fedora Core 9 box, and won’t work on OSX or other platforms. The commands must be run as root.

rm /dev/random
mknod /dev/random c 1 9
chmod 666 /dev/random

For me, this has completely solved the problem of Versions hanging on quit, and I’m back to mostly happy with the product.

Being security conscious he turned on the passcode lock and disabled SMS Preview.

This screen shot shows the SMS preview disabled:

This enables a mandatory passcode. If a message is received during the passcode entry or while the screen is locked, a generic message of “New Text Message” appears, to prevent viewing of messages without unlocking the phone:

If however the phone is placed in emergency call mode, any incoming SMS messages are previewed instead of presented as the generic messages. Thus all I need to do to intercept the messages from his girlfriend is to place the phone in emergency mode and wait 30 seconds for the next sickly sweet message.

This was in iPhone version 2.1 (5F136), the currently shipping version. Since I have no access to beta of non-released firmwares I can’t test to see if it has been fixed since then. For those how care, this is bug 6267416. I don’t have much hope for it being fixed soon, because my security bug 5368148 from July of 2007 is still marked as open, and still unfixed in 10.5.5.

Send questions via email

Those of you solely interested in my sales number can check them here.

Background

I have been writing Objective-C code since the early days of NeXTSTEP (20 years). I work as a contractor and it is one of my many “day jobs”, along with system administration for LAMP systems, and a few other odds and ends. I also have to deal with other languages like Java, Perl, Python, PHP, and so on.

Development on the iPhone is almost exclusively Objective-C. You might think that I would have an extreme advantage, but honestly I don’t think so. Objective-C is easy to pick up if you have any C based language development experience, including C++, C#, or even something like PHP.

Yes the syntax is different. Every language has a different syntax, but it is all the same bits and pieces such as loops, conditionals, assignments, functions, etc.

Yes the framework is different, but then again, most of the iPhone UIKit framework is not a one to one match for the Cocoa AppKit framework.

So my Objective-C experience gave me a good start, but I know several people who have picked it up with little work, even when coming in with little to know serious programming experience.

I bought 2 iPhones on the day they were released and gave them as gifts to a client. But with no development path I couldn’t see buying one for myself.

When jailbreaking became a real possibility and tool chains and documentation started to appear, I finally bought a phone for myself and my wife.

I paid tangental attention to the open source tool chain, and then when the October announcement of a pending real SDK came out, I set aside plans to build apps until the real SDK was released.

Becoming an iPhone Developer

As the SDK announcement approached, I began to track programs on other mobile platforms, and began to think of potential applications to write. I also started to make guesses and observations about what the iPhone development ecosystem would be like.

In most cases I guessed right. In quite a few I got it wrong, or I have no idea whether I’m right or wrong. I’ll discuss that more later on. But the important point is that if you want to have any kind of success as an iPhone developer you need to think about the future of the platform and be prepared for the change, because the iPhone ecosystem is still undergoing radical changes. Items like push notification have been off, on, and off-again.

I felt that using my personal iPhone for day to day development probably wasn’t the best route, so I ordered another iPhone, and an iPod Touch as well.

I applied for the developer program right when it was announced. In the first 2 weeks it seemed that no one was approved. Then rumors of approval started to filter out, and the assumption was that Apple was processing them on some fair basis, such as first come first served, or approving ADC Premier members first. Neither ended up being the case. I was in the same boat as many other developers, with no idea why I wasn’t being approved, or what I could do to help the process.

I downloaded each version of the SDK, and worked on some early code.

WWDC

In 2007 I had attended WWDC, but now I found myself facing a dilemma. I still hadn’t been accepted as an iPhone developer, and the primary reason to attend WWDC would be to learn more about iPhone development, and one-on-one interactions with Apple engineers about iPhone development. I sent a note off to ADC about the problem with attending WWDC without being able to develop for the iPhone, and got the equivalent of a “just wait” notice.

At the time the simulator was even worse than it is now. Major features like Open GLES simply did not work in the simulator, and those are the exciting aspects of the iPhone. Given my background, getting to play with a simulated NSDictionary and some simulated buttons wasn’t something I would need help with at the WWDC level.

I ended up not attending WWDC. A friend of mine (ADC Premier) got accepted as an iPhone developer less than 2 weeks before WWDC. It was at this point that I started to really notice what I call the “If the guards have stopped my beatings, they must have stopped all the beatings of everyone else as well.” mentality. His assumption was that since he had gotten his developer accepts, I must have gotten mine. An alternate assumptions was that everyone going to WWDC was getting one.

Unfortunately neither was true. I ended up getting mine a few days after July 11th when the App Store opened. I know people who attended WWDC who got accepted even later than myself.

In general Apple did a poor job of communicating about the process of becoming an iPhone developer, and they are still doing a poor job. And by communicating, I mean they are just not responding to some questions. They seem to be trying to be deliberating opaque in their answers, walking a path around the truth instead of just explaining this clearly.

An example of this is the $99 fee for the Apple iPhone Developer program. Many are still confused as to whether this is an annual fee or one time fee. As I have previously participated in Apple Developer programs, I was expecting it to be an annual fee. But I still see new developers coming to the platform who aren’t sure, and the signup pages for the program are very unclear. They just say “Standard Program $99″.

While Apple does have a press release that causally mentions it is a yearly fee, that doesn’t make it obvious, and one would think that if the question keeps coming up, Apple would be more active in addressing it on the iPhone Developer signup pages. A $99 yearly fee should not be treated like a source of shame.

An iPhone Developer

Once I had been accepted as a developer I went through the process of generating the various keys, getting certificates, and provisioning profiles. I had about 90% of the application written at this point, and I fired it up on the iPhone.

The results were not good.

Because the simulator runs at native Desktop speeds, it cannot be used to get an idea of the performance of your application. And HazMat, which ran perfectly acceptably in the simulator was slower than molasses on an actual test device.

This is another one of those items were there just doesn’t seem to be any kind of pro active communication from Apple, and with the NDA still in effect, I’m discouraged from publishing my own experimental benchmarks that might help other developers get realistic expectations out of their product ideas.

But enough negativity. Let’s talk about the development.

HazMat Was Three Projects

The first project was to gather and clean the raw data. This consisted of some data in spreadsheets, others in PDF files, and other in CSV files. I had to clean up typos, oddities and the such and get the data into a common set of CSV files.

The second project was to take the CSV files and get them translated into a SQLite database. This second project would lead me to discover errors in the first project, and I would cycle back and forth between the two projects.

The third project was the actual iPhone application which then reads the data from the SQLite database and displays the result. This then lead to another revelation about the data needing to be massaged for performance and display quality. About 30 commodities had very poor name choices. For instance Gasoline is also called Gasohol,which while it is shorter is more confusing to United States users. So I had to hand mark preferred names for certain commodities.

I also at this point started using the app myself, and learning more about particular usability features and performance. A redesign was done for the detail display to break it into General Info, Fire, Spill, and First Aid instead of one long page.

I did the icon work myself using Photoshop and Adobe Illustrator. A lot of images were based on inspiration from various Google image searches, and the App icon is based off a set of free HazMat placards done in Adobe Illustrator by Ian Albert. The original icon was a NFPA icon, but I ended up changing it when I pulled the NFPA data from the application.

Many items were pulled from the first version of the application, because I quickly saw HazMat becoming a huge time sink. I perform development and system administration services as my livelihood, and I already knew at this point that HazMat would not only be a niche application, but probably the most niche application in the AppStore, and that meant a very small amount of income for a lot of work. I decided it would be better to get a solid first version out, and then add more sections to the first screen as time went on. That way it could continue to be a part time effort from me, that wouldn’t burn me out.

When it came to sales, I also had in mind a specific idea of how many units of HazMat whould actually sell. Unlike a game or a productivity utility HazMat has a very narrow audience. My estimate was 2-3 copies s day, with maybe a peak of 14 or so at the beginning. I also formed a mental benchmark of 40 per day as the number at which I was wrong about my estimates.

Pricing

Based on my guess of 3 copies a day average, I guessed maybe a thousand copies in the first year. To recoup my actual time I would have to charge what I felt was an unreasonable amount of money. I felt that while releasing it for free would result in a lot of people downloading the application it wouldn’t provide any benefit to me, not even a psychological boost.

While 99 cents was an option, I quickly saw it becoming the “price of crap” in the App Store. Chris Rock tells a joke about minimum wage, in which he points out that

“You know what that means when someone pays you minimum wage? You know what your boss was trying to say? It’s like “Hey if I could pay you less, I would, but its against the law”.

So it is with the 99 cent price. It acts like a signal that the developer wants something, but that the application would be cheaper than 99 cents but Apple won’t let them charge less. I understand the dilemma for Apple. It is hard to make profit on 10 cent internet sales that cost 25 cents to process.

In the end I decided to go with pure signaling with the application. I choose the $3.99 price to signify that the application was a significant body of work designed for a specific audience. I didn’t choose it to maximize revenue, or because I did great research, or to match price in other mobile devices. I choose it as a pure signal. I believed then and now that I would have the same number of sales of the app whether I priced at 99 cents or $7.99. $3.99 just sent the right signal.

I would have considered 99 cents if there was a way to raise the price and charge for upgrades, where I could slowly raise the price as the value of the app increases. But that isn’t an option in the AppStore and likely won’t ever be an option.

So in the final analysis, over the next 3 years, I might make $2.80 per copy, with about 3000 copies being sold. That comes to $8,400.00 over 3 years. That works out to PFV of about $7,500.00 which is about how much time I have put into the application. I’ll probably put another $15,000.00 of work into the application over that same time period. In short this application is a money loser, but a labor of love and a personal itch. A hobby of time.

All of this was capped with me needing to actually get all the banking information setup in expectation of maybe getting an occasional wire transfer from Apple. For such a small amount, I almost wish they would have given me the option of taking it in Apple Store credit.

Submitting the App

Submitting the app took a couple of tries. Learning how codesigning work, and how to use the codesign command line utility is essential, because when Xcode is not setup correctly it will say the build succeeded even when it didn’t sign the application.

The ApplicationLoader used to submit the zip file also produces some cryptic and misleading error messages. On the positive side it does detect several common errors such as incorrect icon sizes, or invalid zip files.

I work on about two dozen Cocoa applications, so I have a common build and deployment script and practice, where all development builds have a DEVELOPMENT version number, and the build process changes the Info.plist in the app wrapper during deployment. Those builds are done on an old G5 that I have stuck in a corner. It regularly builds each svn repository, detects deployment versions, does the app-casting and so on.

However the G5 can’t build iPhone applications, and even though I have a half dozen Intel boxes, they are all in active use and not ready for corner of the closet retirement.

So I built the application by hand, and forgot to change the version from DEVELOPMENT to something else. IIRC, the Application Loader detected this and rejected the binary and explained the version number was invalid. So I changed it to “1″, and then rebuilt. Unlike my other build process I can’t change it after it is built because the Info.plist is one of the signed resources.

I submitted version “1″ and then waited.

About 2 days later my application still showed “Waiting for Upload”. I have gotten the distinct impression that all the pieces of the App Store are not directly linked. It seems common for people to report that their application says “Waiting for Upload” for hours after they upload and “In Review” even when it has gone on sale. Again something I wish Apple would be clearer about.

But after 2 days I figured there might be a problem. So I sent an email. At about the same time, Apple was sending an email to me.

Rejected

The version number “1″ is not allowed.

Apple sent me a paragraph that detailed the requirement for version numbers, which basically requires they have a decimal point. So “1.0″ is valid, “1″ is not. Foolishly I figured that if Application Loader knew that “DEVELOPMENT” was not valid, it would have told me if “1″ was invalid as well.

So I carefully read the rules and resubmitted with a date encoded version number “2008.08.18″. Normally I don’t do this for OSX applications, but with the mandatory review process it is unlikely I will ever have 2 versions submitted on the same day.

So I submitted, and the next day checked. “In Review”

And then waited, and waited.

Again, there is that feeling that maybe they forgot my application. So everyday I would login, check, see the In Review, and then work on breathing and meditation exercises.

Finally in the early evening of August 29th , I got the email that my application had been approved and was now available for sale. It still showed “In Review”, and I couldn’t find it in the App Store. However I took them at their word and quickly worked up the web site, and a simple FAQ, and then waited.

Later that night I could finally find it in the US App Store, but not in some foreign stores. My belief is that again the process is not a instant worldwide single button deployment, but rather a series of loosely connected systems. I went to sleep happy.

The version posted to the store was the previously not allowed version number “1″. I would let Apple know, but who can I communicate with in a timely manner about this mistake that could possible solve it?

The Sales, The Reviews, The Spreadsheet

The next morning I got up and checked iTunes Connect to see whether I sold a copy. Incredibly I had sold 3 copies. Two in the US, and one in Canada. Pretty much right on target. I congratulated myself on my wonderful prediction.

I also started to notice reviews. Two of the first reviews were complaints that it should have been free, and another that it was overpriced that it should have been 99 cents, but that I was basically smoking crack if I thought I would sell any copies at $3.99. After some meditation and breathing exercises I got over it. Rather than sock puppet a bunch of reviews, or post my own railing at their lack of understanding of capitalism and economics, I instead found solace in two items.

First, I clipped their comments down and posted them on my web site as endorsements.

Second, except for those pointless whining rants, the rest of the reviews were from people who like the app, and were mostly giving it five stars. It seems I had hit an app that at least a few people wanted or appreciated. I also got two emails from people expressing their pleasure with the application, and pointing out ideas for future versions.

The next morning, I logged into iTunes Connect to see how I did on my first full day of sales.

91 copies, with 76 in the United States.

That was a lot more than I was expecting. I put it down to a big first day boost, and went back to work on other projects. I was sure that by tomorrow it would be down to my previously mindset value of 14 or below, or at the most 40 or so, all depending on whether it was a quick crash or soft landing down to 3 copies.

Instead the next day it was 86 copies. Still way larger than I expected. Sales continued for the next several weeks to be higher than expected. After 2 weeks I’m still not below 14 copies. Instead I seem to have soft landed at about 25 copies a day. I started watching and graphing the values on a Numbers spreadsheet.

That third day was August 31st. I added up my sales to that point and realized that I had made $389.00 in royalties. While not enough to retire on, it was still a pleasant surprise. That meant that I should be getting a check for my two-ish days of sales in August. From other posts I had heard that Apple takes about 10 days to settle on the sales, and then about 10-15 days later actual transfers the money. The idea of getting a “check” at the end of September helped sooth the logical part of my mind that said this was not a business project but rather a hobby.

The total for the first 14 days is $1,541.23 of which about $278.43 is in miscellaneous foreign currencies that aren’t large enough for Apple to perform the wire transfer.

My expectations were and remain so low, that each day I wake up to gleefully check iTunes Connect for sales figures.

The Check

On September 10th, I got the email from Apple indicating that the actual financial reports for August payments were ready for download. I expected that there might be a small discrepancy, but was interested in finding out how close to the $389.00 amount the payment would be.

The answer was a bit depressing. No check. No payment, $53 US sales. And onesie-twosie sales in other regions.

I sent a note to the email address for the payment people. For all my previous moaning above about the lack of communication with Apple regarding the development side, let me say that the payment and contracts people are the exact opposite. Every email I have sent them has been very quickly replied to, and questions are honestly and completely answered on the very first response. That group seems to have the process down to machine like precision.

I had made 2 incorrect assumptions. The first is that August meant 8/1 to 8/31. Apple doesn’t pay on calendar month, but rather on financial month (2008 2009). The contract and payments group was kind enough to send me the exact financial calendar they use. Not only for this financial year, but next financial year as well. For August of this year, the financial month was 8/3 to 8/30.

But even with the calendaring mistake I sold more than $53 worth on 8/30 alone. My second mistake was in thinking that the delay from 9/1 to 9/9 in getting reports out was to settle all the outstanding credit card charges, since Apple doesn’t charge credit cards at the same time the end user downloads HazMat.

Instead that time is just for them to reconcile all the transactions. So downloads on 8/30 are transactions, but only some of them are financial sales. Those people who got credit card receipts via email on 8/30 count, those that Apple didn’t charge till 8/31 or 9/1 don’t count.

In the end, I’ll still get paid, just at the end of October instead of September.

Bullets in Hindsight

• I didn’t make a lot, but more than I thought, and it is nowhere near as I expected.
• Niche apps can sell on the store. Even though Apple is promoting games, there are people who want other items.
• Was it better to write something niche that isn’t likely to be duplicated, or better to write something everyone can understand because it isn’t unique?
• I was in books. This meant I stayed on the front page longer.
• The iPhone shows the first 5 apps without scrolling. iTunes shows the first 21. Moving to number 6 didn’t decrease sales. Moving to number 22 did. More people buy on the desktop than on the phone. However I dropped to #22 on 9/11, so can’t be sure if that was the contributing factor.
• Some of the people who downloaded my app probably have no interest in hazardous materials, but found the application to be appealing because of the sense of a source of “secret” information.
• Getting listed at the end of the month sucks.
• You will get bad reviews no matter want. Don’t let the bozos get you down.
• The NDA didn’t stop me from writing code, but it does stop me from sharing 20 years of experience as it relates to this platform.
• There are lots of reasons to sit and whine about iPhone development, there are a thousand reasons to pound out the code.