How we judge Shipaton
A look at the Shipaton judging timeline, screening process, judge scoring, and what your submission needs to make it through.
Perttu LähteenlahtiShipaton is most likely, at this point, the largest mobile hackathon in the world. Across the last two years of Shipaton and Shipyard, around 2,000 apps have made it to the app stores and have collectively made $7,808,524 in revenue. And we've handed out a total of $545,500 in prize money. Awarding the right projects requires a really good judging process.
Every year we outline how we judge apps, and every year we get feedback from participants who didn't know about some criteria, or who feel our judges didn't evaluate their app thoroughly enough. This article is all about explaining in detail how Shipaton judging actually works.
The timeline for judging
Before we get into how the judging process works, it's worth looking at the timeline of the hackathon. Shipaton runs every year from August 1st to September 30th. You can register even before that, and you can submit your project anytime during the two-month period. We even encourage it, since you can keep editing your project right up until the last minute. Despite this, submitted projects still bunch up sharply near the deadline.
Submissions stay steady for weeks, then spike hard as the deadline gets close.
In 2025 25% of Shipyard projects were submitted in the last 3 days!
We've considered judging the apps that arrive before the deadline, but because you can edit your submission until the very last minute — and because a large share of apps only show up in the final stretch — we haven't done that, and we won't. It would take some of the pressure off reviewing every app in time, but it's just not something we can do reliably. As a result, the window for reviewing everything ends up being really tight.
Reviewing up to a thousand submissions in a week
After submissions close, we have just two weeks to do all of the judging. That's because Shipaton winners get invited to RevenueCat's App Growth Annual in October, just weeks after the hackathon wraps. Since winners can come from anywhere in the world, we want to give them enough time to sort out their travel plans.
Last year, that meant reviewing a thousand projects in a week for everything to go according to plan. Reviewing means going through submission videos and descriptions, and downloading and testing the apps.
The judging process
Shipaton judging works as a funnel. Thousands of submissions come in, and each stage narrows the field, so that by the time a judge sits down with an app, they're choosing between a handful of genuinely strong contenders instead of wading through everything. Here's how a submission travels from "submitted" to "winner."
Judging timeline
From submission to winner in nine days
Stage 1
Build window
Projects can be submitted and edited throughout the event.
You can submit early, then keep improving the page until submissions close.
Stage 1: Intake filtering (October 1st)
You submit your project through Devpost, and once the hackathon concludes we export those project submissions to our internal tool. This tool allows us to quickly verify valid project submissions, meaning they have:
- Been submitted to the App Store or Google Play Store, and there's a link to the store page. This year we're validating other stores and web purchase integrations as well.
- Added a valid bundle ID or package name, which we can use to check that the app has the RevenueCat SDK integrated correctly
- All required submission fields have been answered
- There's a video submission of the app
- The required marketing materials for Times Square are included (app icon and screenshots)
Sadly, but unsurprisingly, a lot of project submissions get filtered out at this point. So be very careful that you've added all the necessary information in your submission.
During this first judging phase we also tag each project for the categories it is eligible for. This is based on what you've written in the category-specific questions. If you left the question for the RevenueCat Peace Prize empty, your app won't get judged for that. Remember to answer all the categories your app is targeting in the project submission. Based on these answers, we split the projects between the prescreening judges.
Stage 2: Prescreening
After intake filtering is done, every project entry gets assigned to a minimum of two RevenueCat screeners. These screeners have only a few days to go through all the apps assigned to them and score them from 1-5 in all the categories the app is targeting. To do this, they are required to do two things:
- Watch the first 2 minutes of the submission video
- Read the project submission and the answers
Notice that prescreeners are not required to download the app to judge it, but they quite often end up doing that. Everyone at RevenueCat genuinely loves testing out new apps and seeing the SDK they worked on in use, and the judging process is a way for us to interact with our customers.
Another takeaway is what you should focus on in your video. Not all of us are expert video editors and storytellers, and the screeners do not expect that of you. What we expect to see in the first two minutes of the video are these things:
- What your app is about, e.g. the elevator pitch
- Your app in use
- How and why your app is targeting the different prize categories
The practical advice you should take from that is: don't try to jam your app into every prize category. Focus on the ones where your app makes the most sense. Explain clearly what your app is about, and show how your app works (record the screen, for example).
You can, of course, make a longer video than two minutes, since the videos become public when winners are published. Just pay careful attention to the first two minutes.
Stage 3: Judge scoring
Now the apps reach the judges. You can see the judges we've selected for Shipaton on the Devpost page. We don't tell which judge is judging which category of apps, so that no one tries to reach the judges directly with their app.
For every submission assigned to the judges, each judge must at minimum:
- Read the entire description
- Watch at least two minutes of the required video
- Review all of the screenshots
- Score the app from 1 to 5 (5 being best) in each category
Judges are also welcome to download the app from the store and try it in person (it's encouraged), but not required. For this reason it is important to state in the first two minutes of the video what your app does, and what categories you are targeting. This also applies to the Devpost submission description. Once everything is scored, each judge nominates their top apps per category.
Even after all this filtering, the categories add up: across everything we run, close to 100 apps still make it to this final round of judging.
Stage 4: Final winner selection (October 8-9)
For each category, RevenueCat (or the category's sponsor) meets to deliberate and pick a first, second, and third place from the judges' nominees. Before anyone wins, we go back to the source material one more time, reading the full description, watching the video, and reviewing the screenshots, and at least one RevenueCat developer advocate downloads the app to confirm it matches what the video shows.
After this, the winners are finalized, and we reach out to first-place winners to start coordinating their flights to App Growth Annual in NYC. The app icons and screenshots we asked for earlier are then used for the marketing we do about the winning apps.
A note before you build
This process is subject to change, and it gets a little sharper every year. If anything here is unclear, or if you have feedback or questions, let us know. The whole reason we wrote this is so that no one walks away surprised by how their app was judged.