The team at SEOmoz has been hard at work this week, smoothing out a lot of the initial bumps we’ve seen with our beta launch of the new web app. We anticipated the app would be popular, but I don’t think any of us were prepared for just how many keywords needed rank checking/grading and pages needed crawling/error-checking. Our queue to fetch rankings/crawl URLs had a backup of multiple tens of thousands of requests all week, and the dev team’s been slogging away on parallelization, separation of queing stages and other fixes.
Our next big release is scheduled for August 25 (possibly the 26th depending on how repairs go) and we’re all crazy excited (and more than a little nervous, sleep deprived and caffeinated). Feel free to start marking your calendars; I know we have 🙂
But, today, I’m here to talk about (and ask about ) the future of the web app. We’ve got a nearly endless list of features & functionality we’re hoping to add to the web app in the weeks and months to come, and we need your help in priotizing what YOU care about. To start, I’ll share two lists – the first is our “quick hit” list of items we’re planning to address in the next 2-3 weeks (some will even be in time for our “big” launch on August 25th). The second is some larger concepts we’ve been noodling around with that may take a few months to get in. With both, we’re hoping you’ll give your $0.02 and help us prioritize which items to concentrate on.
Quick Hits List
#1 – Printable Reports (DOC & PDF)
We’ve heard from a lot of users already that they’d like the ability to export the crawl diagnostic reports, on-page summaries and report cards and ranking data into DOC or PDF files to be integrated into internal or client reporting. Luckily, this is a feature that’s early on our roadmap, possibly as soon as September.
#2 – On-Page Optimization Interface Tweaks
The on-page analysis section has already garnered a lot of kind words and hopefully helped many of you improve your targeting for some easy rankings wins. However, there’s a few tweaks that folks have suggested to help make it more usable, including removing the “fix” level of difficulty label on elements that are already completed and offering a way to re-order the recommendations to show those that are incomplete at the top.
We’re also working on ways, in the longer term, to help make this page shorter and the information more quickly digestable. Look for some interface experiments coming soon.
#3 – Adding Issues to Crawl Diagnostics
We currently track 20 unique crawl issues (split between errors, warnings and notices). Some other items we’ve considered tracking include:
- Use of meta noarchive (notice)
- Pages with display: none in their CSS (notice)
- Pages lacking analytics tracking code (warning)
- Pages that return any response code outside those we already track – 200, 301/2, 40x, 50x (error)
- Pages that redirect through more than two chains (warning)
- Pages that serve a meta refresh (notice)
- Pages that redirect with javascript (warning)
If you have additional items you’d like to see in the crawl diagnostics, please let us know!
#4 -“Ignorable” Crawl Issues
Some of our members have noted that they’d like to be able to “ignore” an issue and have it exist only in an “archived” issues section. We think this is a great idea, as there can be times when we catch a 404, duplicate content, robots blocking, etc. and it’s not a problem for your site but an intentional move. When this happens, it can be frustrating to see the continued error/warning message, so an archiving system might be ideal.
We’re still working on the concept of how to implement, but an “ingore all issues of this type” and a specific “ignore this issue for this URL” are currently on the roster.
#5 – Bulk Keyword Import System
Today, it can be a bit frustrating to add more than 5-10 keywords and labels at a time. We’d like to build a system that lets you upload a CSV or paste in rows with lable data included in a consistent format to make bulk insertion and labeling easier.
Big Ideas for the Future
Although we’ve amassed literally hundreds of ideas for upgrading and adding to the web app’s featureset, we’re really excited about a few key ones that have many mini-features inside. These include:
A) Integration with Google Analytics
One of the projects we’re most excited about is integrating with Google Analytics (and later, other packages like Webtrends and Omniture). You can see some of our early ideas below in wireframe format (these ARE NOT finished designs by any means, just illustrations I made in Flash).
We’re keen on the idea of having some stacked are graphs to help you see when traffic from different sources vary, and help to measure indexation via the chart below. Splitting out social traffic by using a set of referrers (ReadWriteWeb does a good breakdown of sources) to filter also struck us as being a great feature.
From there, we’re also bullish on including data about specific keywords alongside rankings, keyword difficulty scores and estimates from Google AdWords:
With this data, we think we can calculate some cool metrics around the potential opportunity of a given keyword, though this will, obviously, require some testing and refinement.
B) Crawl Depth Analysis
We’ve long wanted a way to visualize a site’s internal link structure and know how depth of pages from the homepage might actually be influencing crawling, rankings and traffic. With the custom crawl & crawl diagnostics system, we believe we can architect this into the web app’s dataset (though it’s unfortunately non-trivial to do so). You can see a very early wireframe below:
This is one of our more ambitious projects, but we’d love your thoughts about whether it would be valuable/useful for your campaigns.
C) XML Sitemaps Builder
Building an XML Sitemap can be a pain, even with some of the specialized software out there (though we at SEOmoz are big fans of John Mueller’s GSiteCrawler). Since the web app is already crawling your site’s pages, it only makes sense that we could construct an XML sitemap, plug into Google Webmaster Tools’ API and help you verify the sitemap and make custom tweaks based on what you want to include or exclude.
D) Keyword Research System
A relatively obvious next step would be the addition of a keyword research tool. We’d like integrate the functionality of the keyword difficulty tool’s analysis along with data from Google’s AdWords API. This might help you choose which keywords are most likely to produce value for your site and deserve some content/targeting in SEO.
E) Historical Link Analysis
One feature we hear demand for all the time is historical link information. We’ve actually got the data already stored from previous indices, but in testing retrieval, we’ve found that numbers can really bounce around due to the massive amount of noise in the “not-so-awesome” parts of the web (spammy sites, scrapers, etc). Thus, we’re looking into ways to scrub the data a bit before building this system (possibly by using our metrics to have the option of showing only mozRank 2-3+ pages that link, which tend to be relatively high quality). This work may take us into November or later, but we’ve got our fingers crossed that it can be in the web app by year’s end.
The wireframes above are just some initial concepts. We’d also really like to be able to show you pages/sites that were linking to you in a previous index but aren’t any longer or those that are newly linking, too.
F) Social Media / Link Monitoring System
Finally, we’ve got a project to turn some of the early work from Blogscape and our Social Media Monitoring prototype into a more robust, fully functional system. Our goal here is to provide a list of all the pages, tweets, blog posts and links that your site acquires in a more real-time type environment. So many of us are constantly doing Google Blog searches and Twitter searches and looking at our referrers via analytics that we thought it would be great to combine all that data in a single repository so you can keep up to date on what the web is saying about you (and, more relevantly, how important each of those sources are).
We’re still at the nascent beginnings of this work, but hope to have some wireframes to show in the not-too-far-out future – possibly in the next feedback request post.
Just for fun, I thought I’d include a poll regarding these “big” ideas and see which you’re most excited about:
With our next big launch just 9 days away (yikes!), we’re all working hard to make the web app and the many other pieces that are releasing better, faster and more stable. However, we’d love your opinions and will certainly use that feedback to improve, if not next week then in the future.
Also – as we move forward, we’ve decided to be more open about our product development and roadmap (as part of our commitment to being TAGFEE), so you can expect a post every few weeks or so detailing some of our ideas and asking for your thoughts on what to build next and how to improve.
p.s. If you haven’t tried the web app beta yet, give it a spin – it’s PRO-only, and some sections are a little slow, but by building a campaign now, you’ll have more historical data and trends to compare over time as the app improves.
