We’re happy to announce that EventVue will be delivering the online community for DEMOfall 08!  We’ve been working with the DEMO team to bring the EventVue tools to DEMO so that the DEMO presenting companies and attendees can get the most of the networking opportunities at DEMO.

DEMO is the world’s premier launch platform for the hottest emerging technologies. DEMO brings together the top media and investors to see the latest innovations unveiled for the very first time and we’re proud to be powering the person-to-person connections that happen at the event.   Check out the DEMOfall 08 EventVue community and get a peek at who is coming to DEMO.

Of course, not all of us will be able to attend DEMO so if you want to get the inside track on what’s happening, follow the special twitter account we’ve set up, @demofall08.  This account will republish all the tweets of the registered DEMOfall attendees during the event.  We expect it to be a neat way to hear about all the cool companies that are presenting at DEMOfall in real time from the people who are actually there at the event.

A few months ago we announced our new chatter functionality.  The concept is simple.  Conference attendees add their twitter accounts to EventVue.  We then find their tweets, aggregate them within EventVue and republish them on twitter.  This provides a powerful way for people to follow the conversation that is happening at an event.  We’ve gotten great feedback on our chatter functionality and our auto-updating screens have been quite popular as well.

While the idea of aggregating tweets is simple, making it work isn’t quite as easy.  Here is a behind-the-scenes look at EventVue and the technology that makes it happen.  It’s the story of how a company called Gnip came to our rescue at just the right time.

Our first prototype

For our first chatter prototype we used a simple script that fetched the tweets from each attendee one at a time.  The script ran once a minute and gave us a great proof of concept.  We only had to demo it working with a couple dozen people.  It was enough to get people excited about the feature and prove that we were onto something.  It quickly became obvious that the method we were using for our prototype wouldn’t scale.  We were querying twitter non-stop since we needed the conversations to be displayed in real time.  We quickly grew from dozens of queries a minute to hundreds of queries a minute.   With each request taking several seconds to process, it became harder and harder for us to present results in real time.

Our second attempt

Then we remembered that Twitter provides a feed of all your friends tweets.  Ah-hah!  All we needed to do was follow everyone who added their screenname in EventVue and we could get their tweets with a single request to twitter.  Brilliant, right?  Not quite.  It turned out that the feed from twitter was missing about 50% of the tweets and it progressively got worse as we started following more and more people.  People started complaining and we we were forced back to the drawing board once again.

Getting desperate

We went back to our first prototype except this time we used Summize (now a part of twitter).  Their API is about 7 times faster, doesn’t have any published rate limiting, and most importantly allows complex queries up to 160 characters long.  This meant that we could fetch tweets from a dozen people at a time.  We turned on multi-threading, sacrificed a CPU and started pounding Summize as hard as we could!  This enabled us to process tweets from several thousand people and display the results within just a few seconds.  Not bad.   It still wasn’t ideal, but we had exhausted every other option.  Everything was going smoothly until I got this (somewhat inevitable) email from Greg Pass at Summize/Twitter:

“We notice you’re using the Twitter Search API — that’s great! Unfortunately, you are so exceeding our rate limits, that all of your requests are being refused.  You’re currently at ~34 requests per second. If you slow it down to 1 request every few seconds, you’ll be back in action.”

Bummer!

That’s around the time when I heard that this new service called Gnip had added twitter support.  Gnip promised to “make data portability suck less”.  More specifically, they promised to take away my never ending battle with twitter.  Needless to say, they had my attention.  I started playing with their API and immediately feel in love.   Instead of having to continually ask if there were any new tweets, Gnip offered to watch for me and to let me know when it came across any tweets that I cared about.   It didn’t take much of a decision for us to switch to Gnip.  A couple dozen lines of code later and Chatter was back — better and faster than ever!   Gnip gave me the push architecture that I desperately needed, freed up our computing resources and ultimately saved us a huge amount of time and money.

For anyone who finds themselves in a similar predicament, I can’t recommend Gnip enough.  Gnip has a rock solid service and a team that has been incredible in helping us make the transition.

We’re highly indebted to the guys at Gnip.  This post is really just to say thanks.