This year’s WWDC Keynote is only hours away and with registration completed, you can feel the excitement in the air in San Jose. Thousands of talented engineers are eagerly awaiting for this year’s announcements. Let a week of coding begin! Have a great week everybody.

So you have built your great new wine selling site. You made sure you have used only the best practices. You invested time in making sure your software engineers used the best frameworks available to them. Your UI engineers have ensured that your site is fully responsive and will provide your users with the best possible user exprerience on any device. Whether it is a mobile phone, a tablet, a fridge door with internet connectivity or even something as exotic as a desktop computer – they made sure your site is accessible and designed to ensure optimal performance. But what if you are a bot? No really what if you are a bot? A search engine bot – say like googlebot or yandexbot? After all you want your users to find your site on their favourite search engine, so you made sure that all your links are crawlable and provided with a reasonable robots.txt file? But are you sure you haven’t provided far too many links?

So your site sells wines from all over the world. Imagine a global wine selling site, where every vineyard can sell directly to its connoisseurs. Your site allows your users to:

– Search for wines by name, or colour (for simplification reasons)
– View wine prices in local currencies (25 major currencies)
– View wines in their local language (20 major languages)
– And sort these results by price

Clearly you anticipate that the site will be a massive success, hence why you made sure caching is used properly to ensure optimum performance. But how much can you cache?

Let us assume that you have managed to sign up 1,000 vineyards from around the world. And all of these sell three types of wine (white, red and rose). So your site can sell 1,000 x 3 = 3,000 unique bottles of wine. Each of these bottles comes with a great description, ratings and various tags. Let us assume that each wine has 200KB of data attached to it. So far your site can actually return results of 3,000 bottles x 200KB = 600,000KB (600 MB) of data. So great you can cache all of that and your site will be super fast. But what about the currencies and sorting? Ah yes well that will create more unique cached result sets. Actually a lot more! 20 languages x 25 currencies x 2 sort directions x 600 MB = 600,000 MB (585 GB). Can you still cache all of that? No you can’t. But then you most likely don’t need to. Most users will not convert the prices nor change the sorting too often. You can afford to produce these result sets when needed and cache for a short time.

What about bots? Have you made sure that all your links have rel=“nofollow”? Yes all your A tags have that attribute, but what about your input select tags that you included for your mobile users? These cannot have rel=“nofollow”. And that will cause bots to crawl your site for all of these extra links that don’t really alter the results sets and don’t really add any SEO value. Initially your site will perform fine but over time it will start to buckle a bit. If bots are finding all your currency and order parameters in your URLs then your servers will start to cache slowly more and more data. And because it is highly impossible you will have 1TB of RAM you will start running out memory pretty quickly. Which means your system’s page file will start coming into use and that’s when your site will really slow down. Well until of course bots like google realise this and slow down their crawl rate to allow your site to catch up or maybe they don’t? Some bots, like Yandex, will actually do 20-30 simultaneous calls to your site. Can you imagine the load?

So please make sure of the following:
– All your non result altering links (sorts, currency conversions, locales) need to have rel=“nofollow”.
– If you need to provide a select type of link options, then use javascript to construct them. Hence not allowing the bots to crawl them.
– Upload an appropriate robots.txt file to your site. Ensure you exclude params in them and even set the frequency of querying. Some bots, like Yandex, allow you to slow down the crawl by providing extra params in your robots.txt file.

User-agent: Yandex
Crawl-delay: 4.5
Clean-param: curr&rad&locale

By adding the above statements to your robots.txt you are telling Yandex to allow at least 4.5 seconds between calls to your site and to ignore the specified params. This doesn’t mean that your site will be crawled every 4.5 seconds.

I hope this will help you to not allow bots to control and “stretch” your site’s resources.

I love my Sonos Speakers and I love my Apple devices. But most of all I love the simplicity of AirPlay. Unfortunately I hate the Sonos software. It’s difficult to use and doesn’t integrate well with other services such as Spotify. 

So why not use AirPlay directly from my iPhone or iPad to play music to my Sonos speakers? Well the answer is simple. Sonos chose to not include the necessary hardware and software components needed in their speakers. I don’t really know why, except for the cost of licensing from Apple. For those of you waiting for native AirPlay support from Sonos, that won’t happen. Well not for existing systems anyway. AirPlay does require hardware components as well as software to work.

Enter AirSonos…

AirSonos is an open-source free server that adds AirPlay support to your Sonos devices on the network. The only catch is that you will need to have your Mac running if you want this work all the time. But small price to pay for AirPlay!

So here’s a step by step guide to getting up and running in no time:

1. Download and install Node.JS (a JavaScript open-source, cross-platform runtime environment for server-side applications). This also include ndm (a Node.JS download manager – needed for installing and running AirSonos).

Downoload the files here (Mac OS X Installer (.pkg) – Universal is what you need for Macs)

2. Once downloaded, install the package. Make sure you use the default settings.
3. Start the Terminal App.
4. Type sudo npm install airsonos -g (enter your account password when prompted). The install will take a while (note an Internet connection is needed).

5. Once that is completed, just type airsonos and voila, you will see all your Sonos devices visible in the prompt. Note that this must remain running if you want to have access to them from your iDevices as well as any other Macs.

6. Now open your AirPlay menu on any iDevice and you will see your Sonos players there (might take a few seconds).
7. If you want to run this more easily then simply create a new plain text file in TextEdit (make sure you switch to Plain Text (in the Format menu). Type airsonos in the document and save it on your desktop as airsonos.command. Then grant this file execution rights from the Terminal (navigate to your Desktop folder) and type
chmod u+x airsonos.command

8. Then double click the file and there you have!

You can read and contribute to the project here.

Enjoy your music!!!

Today Apple finally announced this year’s WWDC dates. http://developer.apple.com/wwdc

I now have officially collect my WWDC 2010 badge. So Day 0 of the conference (registration). I was reminded that all the sessions (excluding the Keynote) are under NDA. Therefore I will be blogging live from the Keynote tomorrow.

In just over 24 hours, a few meters from were I am sitting, Steve Jobs will be showcasing the next greatest technologies from Apple. It is amazing what kind of attention such an event attracts. It is almost like a pilgrimage for a lot of people. Numerous people of my flight to San Francisco from London are attending as are hundreds of guests at my hotel. Right now (1 hour before registration opens) there is a handful of people waiting to collect their badges. As I am sat on the steps of Moscone West on this fresh San Francisco Sunday morning, I couldn’t help but wonder, that all these people are affecting the lives of millions around the world in profound ways. They might not be curing cancer or helping the hungry but in a way they are creating technologies that have the potential to do that. All I can say is let the games begin!!!

As expected, Apple today released its fourth Beta version of the upcoming iPhone OS 4.0. As mentioned before I can’t mention the new features/changes due to the NDA. You can find them pretty easily if you just Google. My prediction is one more beta prior to this year’s WWDC 2010 and launch sometime in mid/late June.

Flash here to stay or not? If Apple and Microsoft had their way, Flash will certainly be on its way out. After all, do we really need Flash anymore? For any new piece of technology to be successful, there needs to be a real problem that it can solve. That is exactly what Macromedia (remember them, before Adobe bought them out) did back in 1996 (yes!) when they launched FutureSplash Animator, quickly followed by Flash 1. Back then browsers (IE & Netscape) could just about manage pictures, let alone animation and sound. In 1996, Internet Explorer was at its infancy and Netscape wasn’t that much ahead, while we all had to use really slow modems! Browser rendering engines were limited to basic text & image positioning. Modems couldn’t cope with large amounts of data and scripting languages didn’t exist.

Macromedia had an idea, “what if we could make things move, interact and make sounds but still download quickly?”. And that is exactly what they achieved by building a client rendering engine (Flash Player) that could work inside browsers, yet at the same minimise data transfers by using a proprietary compressed binary file format. It was great, the web became alive. Suddenly we could all play games and watch movies. The arrival of course of home-made videos, YouTube and the cheap availability of fast broadband connections meant that Flash was more in demand than ever.

It wasn’t though only the connections and hardware that became better in all these years of Flash domination, the browsers became exceptionally sophisticated and capable at managing pretty much any type of content. New kids on the block (such as WebKit) made sure that browsers could really and truly tap in to the power of modern operating systems and their graphics stacks. This meant that suddenly there is no need for smaller downloads, nor the need for an extra layer (Flash Player) to achieve something that your browser can do pretty much better and straight out of the box.

As I said in the beginning, any new piece of technology is successful if it solves a real problem. Flash did just that for over a decade. Now that problem that does not exist anymore as browsers have natively taken over, therefore I think it is time for Flash to respectfully leave the party on a high!

>In only 8 days, this year’s WWDC in San Francisco has been sold out! This is the third year in a row that the conference is a sell out. This year though it was in only 8 days and not in a month like last year. I think this clearly demonstrates that the iPhone OS platform is a very serious platform and that more and more developers are seeing its potential. In only 2 years, Apple has fundamentally changed the developer world. Only a small percentage of developers would have ever considered developing apps for the Mac platform using Xcode and Objective-C, but 2 years on, that is not the case anymore. Developers are realising the potential and power of this new platform, and are flocking by the thousands to events like the WWDC.

This of course fundamentally changes the potential of developing not just for the iPhone OS platform but ultimately for the Mac OS platform. Could this be a repeat of what happened to Apple after it launched the first iPod? Before Apple launched the iPod and iTunes, the company was pretty much bankrupt. Users started liking the iPod and eventually thought, and rightly so, that “if their iPod is this good – then their computers must be too”. Apple very cleverly lured people away from PCs & Windows to the simplicity of the Mac. Could this be happening now for developers?

I couldn’t help but wonder, could this really be a time of major shift in developers? Could we see more and more developers shifting to the Mac/Objective-C platform? After all, “if it is this easy to develop for the iPhone OS – then it must be easy to develop for the Mac platform”. It is not easy for any developer to switch to Objective-C, but it is easy enough for a developer with the right foundations. Anyone with good understanding of object-oriented concepts and C could do the jump quite easily. The rest is simply just syntax! A good creative author that writes novels in English can write equally good novels in French, they would just need to learn French.

It is certainly an exciting time for all of us. I think that people who choose to ignore the Mac and iPhone OS platforms are doing so at their own peril.

See you all 5,000 developers in San Francisco in exactly one month from now!

Today Apple released iPhone OS 4.0 Beta 3. Excellent improvements and new features. Due to the NDA I cannot disclose any details. Sorry.

If you are a registered developer – go ahead and download it.