Balancing life

I am torned apart these days. I just seems not to have enough time. I am in need of a life balance sheet. You know those kind of spreadsheet you setup in hope they will help you out.

My main problem is that I want to do everything. I want to live it and enjoy it. Here is a short list of thing I would like to do:

  • Invest more time in training for programming and development skills.
  • Invest more time to make my work job enjoyable for everyone by planning some more activities with my colleagues.
  • Invest some time having fun.
  • Invest some time taking care of my general health, doing sports and some body building.
  • Invest more time helping out open source projects. So far, I am still evaluating which ones.
  • Invest more time in my personal software projects.
  • Invest more time in my family members’ web projects.
  • Invest more time blogging.
  • Invest more time keeping up with the news about software development.
  • Invest more time hanging out with my friends.
  • Invest more time taking care of my girlfriend.
  • Invest some more time with work related out of work projects.
  • Invest some time influencing politician about privacy, copyright and new digital realities.
  • Keep up with the daily stuff.

Right now, I know I am probably spending too much time on gaming. On average, I might spend about 14 hours a week on it. That item should be scrached from the list, but one of my philosophy is that I should have fun everyday. I will not have time to have fun once I am dead.

As for the rest, I just seem not to get enough of each. Am I trying to do to many things at once ? It seems like everyone is doing it. I should be able too right ?

Do you have any tips or cure for me ? I am open to everything.

The joy of Python

Me and Guido van Rossum at PyCon 2009 in Chicago

Me and Guido van Rossum at PyCon 2009 in Chicago

I have been in Chicago for PyCon 2009 this weekend and I must say that it was a tremendous conference. These guys know what they are doing and they are doing it well.

First, the hotel was alright. There is not much to say about it except for the fact that I had to walk 10 minutes to get to the main hotel where the conference was. From what I got, they had to reserve two hotels because they wanted something bigger and they had a contract with a previous hotel.

The food was pretty good. We had a continental breakfast every morning. We also had a great lunch every day that was served fast and professionally.

Most of the talks were interesting. Some of them had either a monotonous speaker or were less captivating than I though they would be. Since all speakers were giving their own free time for this, the quality was high. We were asked to vote after each presentation by dropping a bingo chip in either a green bucker for good, a yellow bucker for so-so and a red bucker for bad. It was an original way of measuring appreciation but my feeling is that way too many people abused it. The logistic of organizing all this was mostly perfect. There was nearly no downtime for people to setup and start their presentation at the right time.

Everyone was given the opportunity to speak during the lightning talks. They were scheduled 12 hours before talking place and all that was required to speak was a laptop to show something and write your name on a sheet. Given my future implication in some python projects, I might do one next time I go.

Another nice thing about PyCon was the Open Spaces. It is a nice concept where people write some topic on small paper cards and place them on a board to setup the time and the place of the meeting. People than go have a look at the boards and plan to attend whatever they like. They had something like 12 smaller rooms that could be used for this. All this was going all around the clock. On Friday night, I played board games with other Python users up to 2 in the morning. It was all fun.

I have had the chance to meet with so many inspiring people. I met with Guido van Rossum for a quick picture. I was so impressed how friendly, open and easy going he is. He attended the conference just like the rest of us without special treatment (from my point of view). Everyone was down to earth and simple, including him. I also had the chance to meet with Python users from Québec.

The community around Python is fantastic. They are a bunch of friendly, intelligent and open minded guys (the PyGirls were drowning in the sea of males) sitting on a diamond mine. There is much to discover by digging below the surface on a social and technological level.

When I left, they were starting their sprints, a developing session, to advance various open source projects while being face to face with people that are often sitting in front of their computer around the globe. Have I had more money and time, I would have stayed to help some of these projects.

I am now back to reality. Time to get those Python skills ready for the next PyCon. See you there.

Getting ready for PyCon 2009

I have bought my flight ticket, my hotel room is booked and my conference fees are paid. I am ready for PyCon 2009.

I am really excited about it. It will be my first real development conference and an international one.

I will have the chance to meet or at least see Guido van Rossum, leaders and contributers to many open source projects I use and I admire: SQL Alchemy, Django, TurboGears and others. I will also have the chance to watch conferences by the developers of EVE Online, a MMORPG game developed with Python.

It is going to be a wonderfull 3 days.

Performance and worries

No matter where it is going, it has to run fast by Hamed Saber

No matter where it is going, it has to run fast by Hamed Saber

Most developers I know are worried about their software performance. They are worried to the point of nonsense. With every decision they will think about, they will immediately balance out the arguments regarding performance whether it is an architecture decision, a technological choice or a nested loop.

This does not make sense on many levels. First of all, performance is more often overrated. A fully functional and correct slow software is always better than a fast buggy or incomplete software. Clients are often asking for highly performing software but in most cases, they are satisfied with a fully working version even if it does not run as fast as they first though.

Developing with a performance mindset often create solutions which contains more bugs, are harder to maintain and are harder to understand and read. Here is a example of this. Compare these two C code snippets:

int i, sum = 0;
for (i = 1; i <= N; i++)
    sum += i;
printf ("sum: %d\n", sum);
int sum = (N * (N+1)) >> 1;
printf ("sum: %d\n", sum);

They both execute the same task. The second one is optimized for performance. Which one would you prefer for ease of maintenance, ease of understandability and ease of reading ? The first one of course.

Another reason why you should not think about optimization too much is because you cannot know which part of your application is or will be the performance bottleneck. You might put a lot of efforts with a specific section of code that is only called once in a while and fast enough.

I am not saying you should not do performance optimization, but that should be the last thing you should thing of. In my mind, I only see two reasons for doing it:

  1. Customers are complaining about speed
  2. You want to gain a competitive advantage (just like the current crop of upcoming browsers)

If you know any other good reason for doing it, let me know.

When you have to optimize your code, the first thing you have to do is use a code profiler to find where is the problem. There are many tools available for this and I am pretty sure there is one for your language. Here is a simple profile output I ran a few days ago for Mercurial.

         1239498 function calls (1237075 primitive calls) in 18.604 CPU seconds

   Ordered by: standard name

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.000    0.000   18.604   18.604 :1()
        1    0.000    0.000    0.000    0.000 ConfigParser.py:106(Error)
        1    0.000    0.000    0.000    0.000 ConfigParser.py:118(NoSectionError)
        1    0.000    0.000    0.000    0.000 ConfigParser.py:125(DuplicateSectionError)
        1    0.000    0.000    0.000    0.000 ConfigParser.py:132(NoOptionError)
        1    0.000    0.000    0.000    0.000 ConfigParser.py:141(InterpolationError)
        1    0.000    0.000    0.000    0.000 ConfigParser.py:149(InterpolationMissingOptionError)
        1    0.000    0.000    0.000    0.000 ConfigParser.py:162(InterpolationSyntaxError)
        1    0.000    0.000    0.000    0.000 ConfigParser.py:166(InterpolationDepthError)
        1    0.000    0.000    0.000    0.000 ConfigParser.py:177(ParsingError)
        1    0.000    0.000    0.000    0.000 ConfigParser.py:189(MissingSectionHeaderError)
        1    0.000    0.000    0.001    0.001 ConfigParser.py:203(RawConfigParser)
        5    0.000    0.000    0.000    0.000 ConfigParser.py:204(__init__)
        3    0.000    0.000    0.000    0.000 ConfigParser.py:211(defaults)
        3    0.000    0.000    0.000    0.000 ConfigParser.py:214(sections)
       11    0.000    0.000    0.000    0.000 ConfigParser.py:219(add_section)
       20    0.000    0.000    0.000    0.000 ConfigParser.py:229(has_section)
[...]
       10    0.000    0.000    0.000    0.000 {setattr}
        2    0.000    0.000    0.000    0.000 {signal.signal}
        1    0.000    0.000    0.000    0.000 {sys._getframe}
        1    0.000    0.000    0.000    0.000 {sys.exit}
        2    0.000    0.000    0.000    0.000 {sys.getwindowsversion}
        2    0.000    0.000    0.000    0.000 {thread.allocate_lock}
        1    0.000    0.000    0.000    0.000 {win32api.GetCurrentProcess}
        1    0.000    0.000    0.000    0.000 {win32api.GetFullPathName}
        1    0.000    0.000    0.000    0.000 {win32api.RegOpenKey}
        1    0.000    0.000    0.000    0.000 {win32api.RegQueryValue}
        5    0.020    0.004    0.020    0.004 {win32file.CreateFile}
       25    0.474    0.019    0.474    0.019 {win32file.ReadFile}
        1    0.000    0.000    0.000    0.000 {win32file.SetFilePointer}
        1    0.000    0.000    0.000    0.000 {win32process.GetModuleFileNameEx}
        2    0.000    0.000    0.014    0.007 {zip}

Using different runs, you can find out which method is called the most, which one it using the most CPU time and which one is the most expensive with profilers. Once you know that, you can start optimizing the sections which are the most problematic.

Once you are done with your optimizations, you have to run your profiler again with the new code and compare the results with the unoptimized version. That way, you can see if you are making progress or not.

I have come to think that while managing your code, you should use a separate branch for the optimized version and the unoptimized one. Since the unoptimized one is probably going to be easier to read and understand, it will also be easier to maintain that is a benefit you should keep.

Stop worrying about performance and enjoy clean code.

Totally distributed

Some new software development models have appear recently. One of them is the distributed model. It can be used to structure a project in various ways, but the founding concept is that everyone owns a full working copy of the project and is not directly tied to a central authority.

In one of my previous post, I wrote about the distributed way of managing source code and what differences it make compared with the common solutions that are used today. Before going any further, here are the benefits I see when using a distributed revision control mechanism:

  • Offline support (commits, branches, merges)
  • You are your own working copy and you can share changes with others in the way you want
  • You are your own working copy and you can break it apart, test it, try experimental things. You are free to play with it.
  • Flexible structure for changes flow (someone has to pull changes from you - aka the Linux model, everyone push/pull to a central repository - aka the central model, someone has to pull changes from you to check that they meet a certain quality and push them somewhere else, etc)

This is great if you are only managing source code, but there are many different things you have to manage for a software project to be successfull. How does it translate in the distributed model ? Some projects like the ones under the Mozilla umbrella are using a distributed revision control software for their code and some central models for other part of the their projects. This is fine, but there are some alternatives.

I discovered some interesting distributed aware tools to complement your distributed revision control software for documentation and bug, issue or task tracking.

It is still early to tell if we are going to see a push towards even more distributed tools like these. We can surely benefit from using these tools in a proper way. I will keep an eye onto it.

Letting go

Holding tight by jbelluch

Holding tight by jbelluch

How hard should you hold onto your ideas? Do you really need full control of everything happening around you? What about trusting people and finding the answers with them?

It is time to let go. Tomorrow is too late. Now is the time.

Back when I started building my first program, I never thought I would enter a religion war zone by making my initial steps in the software development world. There should be a warning sign for this on every tutorials and introduction textbooks about programming.

Warning ! You are making your first steps in dangerous zone. Prepare yourself or you will get sucked in as well.

Everyone seems to know what is best for everyone else. Client will tell you how you should use the session to get a better performing application. Your colleague will tell you how you should you the database to get the best performance possible.  Your boss will tell you which colour you should use to make your application usable.

Developing software the hard way by Cliff Hanger

Developing software the hard way by Cliff Hanger

Let’s start with clients. Most clients I worked with wanted total control over the scope, the schedule and the cost of the project. They know how to do software development. One of the problems, is that they were given that control. More precisely, they were given the illusion of it. Giving that much control to your client is like developing upside down while being tight up. It is not going to work.

I think the first step toward redemption is to educate and explain to your clients what it is to develop software. It is surely not like manufacturing hammers.

One point I would like to emphasis is the requirements. I have seen way too often requirements made of all must-haves. I think it should have a good proportions of must-haves and nice to haves so that the scope become variable and can be adjusted to meet more important schedules or costs.

People in the software industry are known to have strong opinions. They will argue for hours over whether that technology is superior or better than another. They often suffer from the silver bullet syndrome, but the usual answer is it depends. Every solution has a cost, some advantages and some weaknesses. We should learn them and give educated answers about those technologies.