Results tagged “Projects” from Cordinc Blog

Today I shut down my historical weather website. It continually broke because BBC Backstage Weather kept changing the format of their RSS feed. It became too much effort to maintain the site in a working state, so it had to go. Since there was only ever 943 hits on it and the vast majority of those were me, I don't feel too bad. It served its purpose and I learnt quite a bit from writing it. In any case, I should focus my limited spare time on more useful projects.

I have kept weather_report up in case anyone wants to take it up and use it, but they would have to fix it for the new RSS format first.

Update: The weather_report project and the weather archive is no more. The effort required to maintain it was too great, for more information see here.

Today I have released my first Ruby gem: weather_report, version 0.0.1. It connects to the BBC Backstage weather API and gets weather observations or forecasts for thousands of cities worldwide. Don't be fooled by the low version number, it is usable. However, it can only handle the BBC weather feed, requires pre-knowledge of BBC weather location ids and is metric only (all things I plan to fix).

require 'weather_report'
# 8 is the BBC Backstage weather code for London, UK
londonWeather = WeatherReport.new(8)

# to get the current temperature
londonWeather.observation.temperature 

# to get the tomorrow's max temperature
londonWeather.forecast.for_tomorrow.max_temperature

# and much more!

sudo gem install weather_report

There is also a website with documentation here.

I would like to say thanks to the authors of newgem and this rubyforge tutorial, both of which greatly aided in weather_report's creation.

A while ago I wrote a blog post asking is there dumb money on Intrade? The short answer was probably not once the implied interest rate was considered, unless a chain of nearly expired "easy money" trades could be constructed. When I heard that the Goggle app engine supported Java I was keen to try it out. So I ported my manual Intrade interest rate calculation to Java and had it read the Intrade XML feed to check for any contracts finishing soon with high implied interest rates. Unfortunately, the code fell foul of Google's time limit on reading data from other websites. Rather than see the code go to waste, I decided to clean it up and release it. It is a command line Java app with all the code available under the MIT license. It's probably best to just load it up in Eclipse and run the launch file to see it working. Enjoy!

You can download the whole eclipse project here.

Last year I started this by stating I wanted to write a system, put it up online and create a startup around it. Well, 2 out of 3 isn't bad. I'd still like to create a product business, but I'm fairly happy I got Queuesaurus up and running. I partially attribute this result to publishing the goal So I thought I'd write a new list.

Achievable goals (I hope to manage most of these):

• Improve Ruby skills to general professional standard (at present I know it just well enough to do what I want), particularly:
  • Gemify BBC weather component and put it on github
  • Continue working on Queuesaurus issue list (now kept on queuesaurus - viva dogfooding!)
  • Start next web project
• Learn Scala to point of being able to rewrite one of my old abandoned libraries in it
• Learn Blender to point of being able to recreate simple ancient historical architectural scenes (no need for animation or people)
• Decrease net ownership of physical goods
• Blog at an average rate of at least 4 times a month
• Run 5km in 25 minutes

Stretch goals (I'm unlikely to manage any of these without significantly more spare time):

• Get the weather app to a standard where I would be happy to open source it (no one would use it, but it could show what I can do)
• learn Blender and gaming libraries enough to create simplistic graphical MMOs
• Restart abandoned libraries (graph data type, yafal, music library analyser)
• Play poker profitably
• Make enough money from non-consulting activities to cover costs

Update: The weather_report project and the weather archive is no more. The effort required to maintain it was too great, for more information see here.

I often have ideas for little projects, most never go anywhere, but some act as distractions and bother me until I complete them. Recently, I wanted to know what the weather was in London a few days in the past. Easy, I thought, I just find it online. I couldn't. There are many sites with the current weather or forecasts. However, I could not find any with historical weather. So I quickly created one based on my knowledge of BBC Backstage.

I have put the system up here. It is still at an early state, it may change or be shutdown. At the moment it displays only last month's historical weather forecasts.

Recently for a little project I wanted to get weather reports, and being in London my first thought was to use BBC weather. Doing a little searching, the BBC provides a number of RSS feeds for its data (news, weather, etc), as part of the Backstage project. Details of the weather feeds are here. This post gives some of the tips and tricks I discovered using these feeds.

This blog article argues that current online todo list and project management tools don't meet people's needs and are too complex. Judging by the comments of some people at the London Hackers Meetup, this is not an uncommon view. I agree too. This is why I'm trying to steer a user-friendly middle-path between Queuesarus' competitors - a group it seems Ativiti intends to join (yeah, bring it on!).

So how is it going? Well the Beta testing suggests that the functionality is nearly good enough, but that it is still not user-friendly. I'm not sure what to do. While I knew some parts of the system were difficult to understand, I had a plan for them. However, there are problems with areas I thought were fine. As the blog says, it is a hard problem and I haven't seen it done well. More thought is required, and advice is welcome.

It is done, or at least, the beginning is done. I have put the current version of my program on the web, proceed directly to Queuesaurus and tame the work beast! Please note, it should still be considered beta software.

So, what next? Well, I have a massive feature todo list for Queuesaurus (soon to be transcribed to the web version of Queuesaurus in a valiant attempt at dogfooding) and no doubt bugs will be discovered (if you find a bug or have a feature request, add it to the "Queuesaurus Issues" queue). However, that is not a proper answer to the question, I could keep improving Queuesaurus nearly forever. Instead I need to know how far to pursue this or whether to move onto other things.

During the last few months I have had numerous other project ideas, some that could be a business and others that are little more than side distractions. So far, only one has been pursued beyond thought (I'll blog about that later). A few times it has passed my mind to start something new.

The original aims of Queuesaurus were to, at a minimum, keep working until I had something usable and learn about setting up a web business. I think I can just claim to have achieved the first goal, but not the second. There is still a great deal to learn beyond the technical side. I'm not going to leave my consulting contract just yet, nor am I ready to advertise or properly release it, but I think it's worth giving Queuesaurus a bit of time to see how it goes. After that, who knows. I also think I should head out to the occasional local tech event and get the word out. The more people who try out Queuesaurus, the better it will become and I'll get a better idea of its potential.

Anyway give it a try!

A few people have asked me if I have given up on the project and startup idea that was the original motivation for this blog. No, I have not. A large rewrite and a lack of testing has slowed me down. There is a very large list of new functionality and I could of course keep working on it nearly forever before it is "finished". However, my experience has suggested it is best to release early and often after a minimum level has been reached. My project has now reached that minimum level. There is the occasional bug, new functionality to write and the GUI is still awkward - but what needs to be done will be clearer with people (hopefully) using it, or at least commenting on it.

So this would normally be where I put a link...

Unfortunately/fortunately, I am on holiday the next fortnight and to announce it seems like tempting fate a little too much. Instead I will just describe my idea, and in mid-September give the web address. I keep a great number of todo lists around for work and most software development workplaces have bug tracking or project management software. It seems to me there is a middle ground between a simple online todo list (of which there are many) and a complex project management system (again, there are many of these). My goal is something that can be easily used by one person or can handle groups of people working together. Another goal is to make it easy to use for non-IT people as long as they are familiar with the web (thus usable by a wider audience than many of the issue tracking systems that focus on IT people). So there's the plan, soon you can compare theory to reality.

Based on the work I detailed a few posts ago, I have created my own Javascript tooltip library based on Prototype. It is quite simple, opensource and works with Scriptaculous Sortable elements. Click the above link for more information.

Update: based on the work I detail here I have created my own Javascript tooltip library based on Prototype.

Over a decade ago Borland had an IDE called Intrabuilder, which allowed you to create websites in a Visual Basic like manner using Javascript on the server. After leaving postgrad study I worked at a now defunct dot.com called Electrolley for 9 months using this tool to create an online grocery store. Serverside javascript was an idea well ahead of its time. Indeed, javascript on the client side in 1997 was crap too. Now javascript is cool and moving forward (backwards?). With my new project, I need to return to Javascript and the last couple of days have been a hard reintroduction.

I am using Prototype and Scriptaculous to make my website as easy to use as possible. I also found some random code on the web to do tooltips - it almost worked! Applying tooltips to Scriptaculous Sortable elements leads to many problems, and I have just finished largely rewriting the library.

• After moving some sortable elements the tooltips of the moved elements would be displayed underneath the unmoved elements. This is because of the way z-index stacking contexts are created in browsers. In moving an element, Scriptacolous clones it and moves the clone, inserting a new element when the element is dropped. This causes the dropped item to be in a different z-index stacking context than the other items. To get around this, the tooltips need to be drawn outside the moved elements context. This can be done by cloning the tooltip and then inserting it into the document at a point outside the sortable elements' container. I have explicitly put in the id's to which the tooltips is being attached, they should probably be passed in (left as an exercise for the reader).
  
  this._clone = this.tool_tip.cloneNode(true); $("container").insertBefore(this._clone, $("main_content"));
  
  When the tooltip needs to be hidden, just destroy the clone:
  
  Element.remove(this._clone); this._clone = null;
  
• This also means the tooltip needs to be positioned absolutely, so ensure you are using absolute screen x & y coordinates.
• Dragging a sortable element over other elements results in a number of mouse events firing as the elements are redrawn. Many of these events are consumed by Scriptaculous and not passed on. Thus, the tooltips hide and show functions may be called in any order, resulting in spurious and permanent copies (since it is cloned) of the tooltip all over the screen. To get around this just ensure there is only ever once tooltip on the screen at a time, so at the top of the show method put
  
  if (this._clone) { hideTooltip(event) }
  

It seems simple now I've fixed the problems, but it took a couple of days to remember my old Javascript knowledge.

Alpha testing has not gone as well as I hoped. Over the last few weeks I have been adding functionality and I thought the system was quite usable. So as a final step before expanding testing, I sat down with my first alpha tester to watch them use it. I expected them to suggest extra features. I had a long list of extra things I could do, but wasn't sure which were important. The good news was that they only suggested a couple of small things that I already had on my list. The bad news is just watching them use the site made me wince and clearly wasn't the best experience for them either. Over the 10 minutes of the test I quickly filled a couple of pages with issues: what does this mean, what does that do, why is that there, how do I do that?

I can heartily recommend watching someone removed from development trying to use your system. After a couple of months I am too close to the program. I no longer see what is displayed, I just know what needs to be done. I also know how to move around the site. Seeing the site through someone else's eyes was illuminating. Watching them have to move the mouse all the way across the screen to do two related actions made me realise I focus exclusively on one bit of functionality at a time. Seeing them wonder if two similar but unfamiliar terms referred to the same concept made me realise I don't read the text. When they suggested that grey buttons made it look like they we deactivated, I remembered that I meant to spend some time on the design instead of just working on functionality. I have decided that I will not add any new functionality for the next couple of weeks, instead I will focus on fixing up what is already there.

I read this blog entry on common distractions that will delay a startup launch date. Sad to say I have previously done all these on previous projects before ultimately abandoning them. Happy to say that so far on my current project I have avoided all of them.

The last month has been quite a busy one; I went on holiday and work is keeping me busy. However, I need to keep things going and not let large time gaps discourage me, so here is an update.

• Alpha testing going slowly. There are a number of regression bugs from my occasionally drastic functionality changes and these tend to stop things for a day or so (I find heroku is too slow to edit online, so I have to fix it at home and upload). At this point the Test Driven Development people are probably having a laugh at me and rightly so. In my defence I am still deciding how various features should work so there is a great deal of change occurring.
• My design skills are as bad as I thought according to the alpha tester - this will definitely become a problem later.
• I have taken up golf and poker in the last few weeks and I'm spending time learning. It's a great deal of fun, as I seem to have picked up the basics of both fairly quickly. I aim to write more on these in future.
• I must find more time to work on my project! I'm also setting a goal of at least one blog post per week.

I've also learnt a bit about web advertising and SEO through this blog. Up until the Ultrasphinx post very few people visited this blog. Since that post I get around 5 people a day, with a couple of spikes to 10 in one day. Most of these people come from google searches (check it out), only read the ultrasphinx page, and tend not to stay long. However, a surprising number of them visit again. I have to admit to a little fear and trepidation when I first saw the explosion (albeit highly targeted) in readership. I wasn't sure what to write. Should I write more technical articles, as that is what drives people to this blog, or just continue as before. The lack of technical things to write about made the decision for me, although I do double-check my spelling now.

In any case the extra readership hasn't made any difference to my ad clicks or google page rank. Google still classifies cordinc.com as low popularity. No one has clicked an ad since the first week of the blog, earning me a grand total of US14cents. I had originally thought that my project could be funded by ads, but I'm not so sure now. I don't think there is as much money in it as before so I may need to rethink the business model. Perhaps it's best just to get the system finished first.

I reached a milestone in my project progress yesterday. I have just put a very crude version of my app out for alpha testing. Unfortunately this is three weeks later than I had originally planned back in January. At that time I was hoping to do an alpha release over the Easter holiday. About half the delay is due to problems with the place I decided to host the app, Heroku. My app seemed to screw up their import system and for days I could do nothing. It also seems a bit slow - noticeably slower in production mode than my old laptop in development mode. When I get to beta testing I'll probably move to a different host. I can't complain too much, as their service is free and in beta. They are definitely worth checking out for small developments or hosting.

The other half of the delay is caused by the time taken for my expectations to return to their original level. Over the last three months my ideas on what the app should do have expanded greatly, and that week of delay was me trying to add some last minute features. I could have kept going forever before releasing anything. Ultimately I decided it would be best to get something out even if I knew it was a crappy bare minimum. I hope my alpha tester will persist through some trying times, but right now I need a little direction. The app is definitely not as easy to use as it needs to be and I suspect the design is atrocious. The plan is my alpha tester will help direct me to the areas that need most attention, and guide me away from those that are less important.

I have decided that my next milestone will be the end of May for expanded alpha testing. By then I hope to have something that I can show to a small, but slightly larger group of people. Anyone interested?

I was hoping to have a minor status update to announce by now. However, over the last fortnight there have been a few issues with an external supplier - I will detail then when they are resolved one way or the other. In the meantime, this caused me to stop breathing today, at least until I read this.

After having the simple idea (see here) over the New Year holiday, I next needed to decide how to implement it. Many people would say I needed to write design/spec/use case documents first, but for this project I think best to start working and see what happens. It would be easy to draw diagrams and think about problems with nifty solutions. However, the main goal is to DELIVER. I want to get something up and running quickly, get some people to use it and then improve it. I believe the longer it takes to get something up in front of people, the more likely I never will reach that point or go further. Having people use the system (even as testers) is a form of commitment.

As a result I chose to use Ruby on Rails for development - something many people who know me may find surprising. I have never programmed in Ruby before, so I'm learning it at the same time as trying to quickly develop a new system with fuzzy requirements. What happened to avoiding development risks and not trying to do too much at once? If I did a project risk assessment like most of my previous employers this would raise some serious red flags - the project would not be allowed to go ahead. Although, no previous employer would allow my idea to go ahead anyway, they would see it as too small. So why did I make the Ruby decision?

For the last 6 years I have almost entirely developed in Java (and before that a mix of Java, C++ and other languages that will never appear on my resume). I have worked on developing a wide range of systems from the large (over 100 people for a few years) to the very small (just me for a couple of weeks), although mainly for large financial firms. However, I specifically did not want to develop in Java. I may be faster in Java, but there are a number of reasons I thought it best not use it this time. People say it can be heavier to develop, with more boilerplate code, and I can certainly understand that. There are also less decent internet hosting options for Java, nearly every hosting firm provides PHP or Ruby options, but finding a good Java host is harder. However, the main reason for me was that with Java I was liable to get bogged down in the intricacies of the language and various libraries or third party tools. With a new language I wouldn't have time to think further than the next problem I was facing and finding a solution for it - there would be fewer distractions simply because there wouldn't be time to lose focus or investigate interesting side issues.

Choosing the language was difficult. There were so many I wanted to try as I haven't properly taught myself new language since C# some years ago. I have been reading (with interest) blog posts arguing (flaming?) the pros and cons of various web development languages and frameworks. Ruby on Rails seems to have a large following and been around for a while. The basic idea has been copied so many times by other languages (Groovy on Rails, Rhino on Rails, ...) that I felt there must be something useful in there. I rejected all the derivatives because I didn't think they would be mature enough yet and didn't have a large enough community behind them. After all a framework with a backlash must have reached a certain level of maturity! The speed issue was my main concern. At work code execution speed is extremely important and has a direct affect on the bottom line (I work on financial trading systems). However, the performance requirements for my system are far less than at work. Ruby will probably be fast enough - or at least fast enough at the start to see if the idea is worth pursuing. So I decided to just give Ruby a try. If it didn't work out I could quickly move onto something else.

Anyway, I've been writing Ruby on Rails code for nearly the last two months in my spare time - probably only the equivalent of only a handful of workdays in total. Yet I am quite impressed by how much functionality I have implemented. I am probably only a couple of weeks away from being ready for alpha testing on the functionality (the website design is still a mess). So far I have to say that Ruby on Rails has been a positive experience, although I am yet to properly test its performance.

I have nearly always coded for myself - that is, not for money or grades or anything other than my own amusement. In my early teens I learnt to program by writing a Risk-like game in Basic. During university, I taught myself C by writing a program to create imaginary worlds because I felt I didn't learn enough during the "official" C course. Since then there have been many little games, libraries, ID3 taggers, etc. My "current" one I have called YAFAL - Yet Another Financial Analytics Library. I might write more about it later, but at the moment it is suffering the same fate as all the others. There has been no work on it since the new year, it has been superseded in my thoughts by company ideas (I don't think YAFAL could make it as a business).

Basically, I tend to get to the point of solving the problem which initially interested me, and then my attention strays. This has not been a source of concern. I was doing it for myself, so I can happily define the finish point in any manner I wish. When working for other people, I always finish the job. Now I need to go further for myself. If I am to found a company then I can't stop after the interesting stuff is done. I must keep going to release and then continue to support it. Hopefully this blog will help. By merely stating my intentions online and letting people read them, I feel a little more commitment to get them done. Or, at least that is the plan.

So the current situation is I have a side-project, YAFAL, and the startup idea. Actually, I have a couple of startup ideas. One is an original twist on an old idea and as far as I know no one is doing something similar. However, it will be a big project to complete. The other is fairly small, easy to implement, useful to many people and has a dauntingly huge number of competitors. Normally I would desperately avoid competitors. Now I think the small idea is the one to try first. It will be best just to get something up and running, learning the tricks of the trade along the way. As for the competition, I like to think of that joke about escaping from bears - you don't need to run incredibly fast, only a bit faster than the guy next to you. If worst comes to worst it should fail-fast at little expense beyond my personal time and I may think of something better during the experience.