Zooomr photo sharing site, an upstart challenger to Flickr, has been down for the past few days while they roll out “mark 3″ aka version 3 of their site. Looks like they aren’t going to make their target of being back online today, March 15th 2007.
Now I know they have a one-man dev/admin/whatever technical ‘team’ over there and probably very little in capital $ available for infrastructure, so I should cut them some slack. I won’t though. This is very very unacceptable and hopefully they will learn from this.
It should have been possible even with limited resources to have the site minimally available while they were doing these upgrades and then have a small downtime to cut everything over. Even so far as to disable uploads and new comment postings perhaps, until the new db was ready.
I guess this is what you get when there’s rank amateurs at the helm. Oh well.
Let’s see what happens.
Place your bets. Will they be online later tonight? Tomorrow? Will they have to roll back to the previous version? Can they even do that? :-}
Edit (March 16 1045 MDT): Their developer burnt out after 48+ hrs at the keyboard, and at some point was counting sheep. Apparently 48+ hrs in a row after 18-20 hr work days for 6 months can do that to a guy. Yeeesh.
Just to correct, it has been the past 2 days, they took the Beta site offline on the 13th, today is the 15th. They also announced that they were taking the site offline during the change…that is more along the lines of “planned downtime”. They have also been releasing info along the way on the progress (and the snags) that they have run into, which is more than most sites who suffer any type of outage, planned or not, do for their visitors.
Kristopher and Thomas have been talking about the release of MarkIII as a “complete rebuild” of the system. I don’t think that any of the current users are as upset about this current downtime as they are eager to check out the new features. It has been discussed thoroughly on their blog over the past few weeks, if not months. The comments over the past 2 days alone speak for themselves.
All in all, I personally think it’s best that they build the Zooomr core around their ideas now, leaving plenty room to grow, rather than having their ideas outgrow the current system they have (had) in place. I trust that they are taking the steps now, to set them selves up for the long term, rather than having to tweak, patch, and rebuild the system as they go.
At this point, I think it best to let the release speak for itself when it is finalized and then take it from there.
cmiper
March 15th, 2007
Ditto for cmiper’s message. I think you need to chill out. MarkIII is a HUGE release and a complete re-design of everything. The storage and data retrieval scheme has changed entirely. If you knew what was going on behind the scenes right now, you’d freak out. So relax, go to bed, because trust me, you wouldn’t want to go through what Kris and Tom are going through right now. This is a huge migration and it’s going to be amazing when they pull it off.
Raoul
March 15th, 2007
With all due respect, you must give Kristopher and Thomas a break. They are only a two-man team, and are undertaking a monumental upgrade. The amount of data ‘gigage’ they have to deal with must be astronomical. As stated above, this down time was not a surprise to active users of Zooomr. It’s been in the works for quite a while. I personally do no find the down time a big issue, even if they need to extend it an additional day or two. They have been very good about informing us, the end users of their activities and progress. This is much more than I have ever seen from any other company. I’m sure if the Mk III version is as successful as it sound like it will be, they will have funds in the future to hire additional admins and pay for additional computing hardware that will enable them to keep the site up during the next big upgrade.
Regards,
JeffH
Jeff H
March 15th, 2007
Just a quick Q for you. Have you paid for the service Zooomr provides?
Daysleeper
March 16th, 2007
I figured that with enough bait I’d get some comments here. I am well aware that they are a two person team. One person really, since this bit is all technical and afaik TH isn’t the technical type beyond his personal photo workflow.
No I haven’t paid for their services yet. The previous version was far far too slow to be useful to me. I was waiting for the site to get faster, even in it’s MII state.
My main point was that proper planning etc would have eliminated most of this downtime. There is no need to have a near complete site outage for several days just to do an upgrade. One builds it in parallel and builds migration tools to do several passes over the data. One of the next to last passes would be to lock the old site for changes (but leave it open for view-only, and I’m talking more than just blogged photos here) and do the final pass then cut over to the new code.
Just highlights that this is a very amateur operation. Impressive given the scale of it and the lack of staff something this size should have already, I will give them that.
Just a gut feeling but I think they likely should be sitting at approx 3 developers and 1 or 1.5 sysadmins at this point, if not soon after MIII launches. The sysadmins would be the ones with commercial site launch experience and high availability etc experience, that would guide the developers on how to make their pretty code stay up and running thru times like this.
anyhoo, yes I will chill out for a while and watch with amusement as zooomr grinds completely to a halt while Kristopher finally burns out and needs an 18hr nap.
Its not worth your health, btw, to work so hard. Work smart, not hard.
trever
March 16th, 2007
cmiper, Raoul and few others (who are regulars over at zooomr blog) seem to advertise that all zooomr users are happy with the site being down since they have a “good” reason. (I do not find the reason as “good”.)
These regulars they do not miss a chance to respond to any critic made to zooomr. They always say “well Mr Tate is the sole developer and blah blah blah… Give them a chance and …”. This shows how fanatic they are and they do not see clearly
When they rolled out the 2nd release they had an outage again and this time all photos in all blogs were down, so I guess this time the users are lucky.
but anyway the topic here is that they do not appreciate that hiring developers, sysadmins and invest in server equipment will make them transition from armatures to professionals.
Let’s see are they going to finish with this upgrade today?
My Opinion
March 16th, 2007
Umm…I never said anything about being “happy that Zooomr was down”. I was talking about the fact that most people understood, based on the information that was given to them, and the comments that have been made in return.
I am as fanatical about Zooomr as your are Anti-Zooomr, if we are going to base this on ones comments. If you think I come here as a “fanboy”, then I suggest that you go find me on Flickr, Photoblog, Vox, and the likes and see where my loyalties lie before you do such.
If you’d actually read my “regular” posts on Zooomr, you would have seen the criticism in the past as well as the suggestions, but I am certain that you merely skimmed the blog to come back here with some meat to fill your comment with, otherwise you would have also noted that during the last outtage, there was a documented DOS that spanned a few days, where the site was accessible/in-accessible.
I now realize that the OP (admittedly) created this post as a troll, so I won’t be responding again to fulfill his needs. Call it a cut and run if you have to, I don’t care. I’ve said pretty much all I have to, and I am not going to convince someone (who obviously is just using this discussion to draw attention) that the steps that they are going through are acceptable to me.
cmiper
March 16th, 2007
I didn’t explicitly post this to troll. It was a side benefit to get some traffic to my blog. Isn’t that what trackbacks are all about?
Anyways, my point if I am still not making it clear, is that for a commercial web site (and if you take money, you are a commercial web site) should be held to higher standards. The least of which is to not have extended periods of downtime, planned or not, announced or not, users-updated-hourly or not …
I will cut them a very small amount of slack right now, but they had better learn from this if they want to play in the big leagues.
trever
March 16th, 2007
Zooomr was to start offering paid accounts AFTER MarkIII. Prior to that, you could get a Pro account by being a blogger and blogging you photos from Zooomr, no money was involved.
The more you know!
Zwhooo?
March 16th, 2007
Look cmiper,
I am not going to spend my time researching on what you do and what not, and anyway it is not my problem you have the right to do whatever you feel like.
As far as I can remember (from zooomr blog) you have not made any criticism towards zooomr. If that is not true my apologies.
My point is that only a minority of people see outage as problem to zooomr’s service and usually the majority does not raise its voice and you know why because they do not care, if zooomr fails (not that I do care. In a quite afternoon I just said my opinion).
When somebody raise his criticism there is always someone to make excuses. Don’t you find that strange?
Anyway,
Take care.
My Opinion
March 16th, 2007
Hey guys, We are working as hard as we can.
Trever, as part of the rebuild for Zooomr Mark III we’ve completely rebuilt the site from the ground up. Part of this involves completely transferring all photos from the old system to the new system.
We needed to turn off the old Zooomr to ensure that we got 100% of the photos, edits, data, etc. over to the new system.
Part of the conversion to the new system involves processing and storing photos in new and different ways.
Unfortunately this has taken us longer than it should. But Kristopher is working in the next room over and is working his ass off on getting it finished up. It needs to be right before we release it though.
Keep in mind that we are a 2 man, one developer beta site and have never taken a nickel from a single user. Your comment, if they take money is actually wrong.
Yes, it would be nice to have three engineers instead of just Kristopher at the time being and hopefully we will be hiring some new people in the near future.
But right now we are doing the best we can with what we’ve got.
Nobody wants the site up right now more than we do. Feel free to criticize us if you want.
This conversion and our difficulty is definitely a learning experience and hopefully we will be the better for it in the end.
Cheers,
Tom
Thomas Hawk
March 16th, 2007
Ok, so I was wrong about Zoomer having some paid accounts. For some reason I thought that was an option before.
Anyhoo. Yes, please do learn from this. As you grow larger and DO take money, you really really really can not be doing this sort of extended downtime in the future.
Looking forward to seeing the new stuff strutting its stuff, and perhaps porting all of my flickr (or at least the better ones) over to zoomr when that feature is available. Who knows maybe I can get a nickel for a picture here and there.
BTW if you need some sysadmin types to manage the back end for you . . .
trever
March 16th, 2007
It always amazes me how easily people can get the wrong message from a blog posting. Trever’s posting is no different.
The point that was being made was that the plan for upgrading the Zooomr server with new software would NOT pass muster in a commercial/production environment. It wasn’t an anti-Zooomr commentary, but rather was a comment on the lack of planning that went into the upgrade.
In this new “global economy” there is no room for downtime, especially for an internet business such as Zooomr. Gone are the days where you have a maintenance window during the night outside of business hours, there are no “business hours” on the internet, just 24/7 baby. That means that upgrades need much more careful planning, staging, and testing.
In Zooomr’s case, it would have been relatively simple to bring up a 2nd server with the new software to use for migration of the data while leaving the original one online. Several test loads of the data could have been done with no downtime on the production servers, and yes that would include “processing and storing photos in new and different ways”. Certain portions of the original site could have been disabled or restricted during the final conversion pass, nothing elaborate, just the site in read-only mode to ensure that no new data could be introduced during migration. Don’t have hardware for another server? That should not have been an obstacle either, VMWare server is free, anyone can afford that. Comments on Tom’s blog that Kristopher has been “coding like mad” only emphasize the lack of planning, let alone testing of the new software. No one should be coding like mad during a conversion, if you are then you weren’t truely ready for the conversion. Testing is crucial before attempting the migration of data off a live site.
There was no reason for downtime other than a lack of planning. There is no reason that a plan could not have been put together on how to migrate the data. Downtime like this can *kill* a commercial site, especially one that might have less than technical customers that don’t understand (nor should they need to) the technical issues involved. The downtime is nothing more than an indicator that the site operators didn’t understand the technical issues involved either. The fact that Zooomr is NOT a commercial site yet should not be an excuse for the downtime. If you want to become a commercial success you need to demonstrate the ability to operate without incidents like this. Potential investors would not look on this favorably, there is even a good chance that this incident could cause some to walk.
I’ll urge Tom and Kristopher to take Trever’s comments for what they are, not a flame, not a anti-Zooomr posting, but rather the application of his cumulative experience managing commercial sites. He knows what he’s talking about, he knows how to do it right, and if he’s bitching about how you’ve done something you should listen. If he didn’t think the site was worth the effort, he wouldn’t waste his words on complaining.
Cheers,
Camz.
camz
March 16th, 2007
One thing for sure is happening. Kristopher is gaining valuable experience. It will be a while before he can fully assimilate what happened here and what could have been done to make a migration go smoother.
A while longer (usually measured in years and a few more ‘events’ like this) before the experience is able to inform his gut instincts from deep in the subconcious.
That kind of thing is what made me twitch in the first place when I saw what was transpiring. Ahh, youth, inexperience. They are so fleeting.
trever
March 17th, 2007
On a slightly different note, but still related to the zooomr update, is this interesting blog posting and reply by Thomas Hawk re: the stock photography sales angle that the new zooomr site will be angling for.
http://www.jmg-galleries.com/blog/2007/03/16/zooomr-a-promising-future-on-hold/
trever
March 17th, 2007
It’s late afternoon on Sunday and Zooomr’s still MIA. Oh well, hope they didn’t have a catastrophic meltdown.
Obviously something substantial has gone wrong (or many minor things).
Let’s hope that once they do get the site operational again that the majority of the MIII teething pains will be over shortly after that.
Hopefully it won’t be prone to every-other-day db corruptions or grid crashes like another how-do-they-make-money web 2.0 site that I frequent, Second Life.
trever
March 18th, 2007
… and zooomr.com mark II (previous version) is back online as of approx 8pm MDT today. Not sure why it took so long to come to this conclusion, but whatever.
Upload limits have been eliminated totally for now, as a guesture of good will. Hope to see the Mark III site when it’s done. Whenever that may be.
trever
March 18th, 2007