Nov 15 2007
Measuring Blog Success and Formulas for Top Lists
I’m always considering new ways of ranking the Top 100 Australian blogs. I’ve been updating the list for 7 months now, and obviously a fair amount of time goes into maintaining it each fortnight. I’d like the list to have as much integrity, credibility and be as inclusive as possible.
As a refresher, the formula I use takes into account Alexa Australian Rank (AU), Alexa Rank (X), and Technorati Rank (T).
Index = (3 x AU + X + T) /5
Estimating Traffic for the Top 100 Australian Blogs List
Following on from my comparison of Alexa and Compete, these are some of my concerns:
- The favouritism shown by Alexa to blogs of a technical nature
- Blogs on shared domains (excluding WordPress.com and Blogger blogs, which do provide an Alexa rank) or part of a larger domain – perhaps not being accurately ranked
- Is Alexa even credible/reliable enough to factor in the formula (but bearing in mind that the aim of the list is to find blogs which are popular amongst Australians – which is why the Alexa AU ranks is weighted)
- Blogs that have many subscribers, but do not fair well on Alexa are often not (well) ranked
- Technorati Rank for blogs which release plugins or themes being (unfairly) lowered
How is blog success, or popularity measured?
As Shai points out:
But, I really, really, really believe that a blog’s success is not just measured by Technorati authorities, Alexa rankings, Google PageRanks, pageviews, unique visitors, and comments. And yes, not even Awards. 😉
I certainly agree, but in formulating a list you HAVE to use quantitative measures. And obviously, the more automated it is, the easier my job is each fortnight.
Ratified use a combination of Technorati (authority), Google PageRank, and number of subscribers in formulating their top list. The problems that I have with this methodology are;
- that it’s not inclusive. If you don’t use Feedburner, or activate your awareness API, then your ranking is extremely affected,
- that it quite heavily weights Google PageRank, which given recent controversy, is probably not the accurate tool of “authority” measure that it once was – particularly amongst blogs.
I also read a very interesting presentation by Avinash Kaushik (a Web Analytics guru) that he gave recently at blogworld (presentation available at EightBlack). Avinash talks about blog measurement and showcases the growth of his blog Occam’s Razor over it’s history. He also covers measuring blog success on his blog.
While he talks about measures such as visitors, subscriber growth and citations (i.e. Technorati authority), he also talks about “conversion” on blogs. Through a plug in called generalstats, it’s possible to see how many posts have been written, the number of words in the posts, the number of comments and the number of words in the comments (if you scroll down in my sidebar, you’ll see I’ve added these at the very bottom).
From these statistics you can determine your rate of conversion by this formula:
# visitor comments / # posts
(note this is a little hard to compute, because you have to subtract your own comments from the total. I think this can be a little hard to determine with accuracy).
This is all good and well for measuring your own “success”, but conversion is not something that can easily be measured externally.
It’s interesting to note that in July, Avinash formulated a list of top 10 Web Analytics blogs using feed subscribers (E4) and Technorati rank (F4) as the variables. The formula applied (devised by Kevin Hillstorm and adapted by Avinash) is somewhat more complicated than mine 😉
Kevin’s inital formula uses Alexa and Technorati. But I’m digressing.
So, excluding variables that are too hard to ascertain (number of posts, comments etc), some variables that have been suggested for inclusion into the formula are
- Google PageRank
- Age of a domain
- Back links from Google and/or Yahoo
- Technorati ranking
- Alexa ranking
- Feed subscribers
With feed subscribers, as mentioned, this is not available for every blog. So if this figure were to be included and wasn’t available, this would either have to be offered by contenders for the list (honour system, or via some proof?).
Alternatively, I could use a Bloglines and/or Google’s subscriber count. Obviously this has dangers, as some blogs report that Google only counts a fraction of their subscribers, and this figure varies from 10% up to 40% of the true count.
Over to You
Given the concerns that I’ve outlined, and keeping the updates as automated as possible (it’s a labour of love, remember 😉 ), here’s your chance to have a say.
- What do you think makes a blog “successful”?
- What variables do you think should be considered in measuring a “top blog”?
- What formula would you apply?
- What are your thoughts about the credibility of such lists, and how can they be more credible?
I look forward to hearing your feedback.
21 Responses to “Measuring Blog Success and Formulas for Top Lists”
I always thought for a blog to be successful a lot of people would have to be reading it which is why my inclusion in the Aussie top 100 is a bit of a nice surprise to me because my readership isn’t that high I don’t think.
I usually do work on my networking though and this is where Alexa and Technorati come into it.
Feedburner is an interesting one too considering those numbers change all over the place (Darren Rowse at Problogger has written about this).
I’m no good with formulas so don’t really have a suggestion there.
It’s a tricky one Meg because I don’t think there’s an easy answer but I appreciate your labour of love.
Good luck.
I love it when you talk geek 😛 7mths? Has it really been that long.
I for one had never thought about it much beyond being very thankful that you went to the trouble of putting the list together and updating it. Knowing that its hard to find anything too accurate. It and you have done big things for the Aussie blogging community, I would never expect of you to do anymore. But now I see your side a little, and have wondered the same things (just with out all the scary formulas 😉 )
I’m like Jen, I am a little surprised, and feel a little guilty for my blog (though it is slipping now) being so high on the list. The rank is more my whole site which is 3yrs old, rather than my blog with is only about 6mths old.
Sorry, can’t offer any suggestions other than we have already discussed. Hopefully someone else can, or we can all make you feel much better that you are already doing a wonderful job with what you have to work with
I’ve come to the conclusion that Feedburner subscribers, where available, are the best proxy for blog success.
Take my blog — my “authority” on Technorati has dropped from about 150 to about 120. My Alexa ranking has dropped from about 225,000 to 450,000.
And yet, during that time, my subscriber base increased from 350 to 850.
Ultimately, the number of people actively reading the blog is what probably matters most.
Good luck with your rankings!
Meg, I don’t know about you but my RSS reader is choked. I have 3-4 blogs I read daily and then I scan about a dozen others. I think the measure of success of any blog is “engagement”. That’s not just comments, but how it influences those who read it. How many people refer to it, depend on it and enjoy the read. I know for a fact that if guys like John Battelle and Seth Godin stopped posting, I’d be disappointed. Same goes with Avinash.
I’ve printed this out to give to the figure man in the house. He LOVES formulas where as I love a good Caesar Salad. He can do his mumbo jumbo furrowed brow thing, tuck the calculator behind his ear and the pen up his nose (this man seriously worries me when he gets excited) and we’ll get back to you if he falls upon a magic formula.
As for the non technical me….I am thinking that reach, rather than subscription, is a measure of success. I am subscribed to many journals but I never read the buggers. The ones I do, I read because I really appreciate what they provide me.
This is a hard question. I agree the list needs to be credible and offer a true reflection of success. I also think we need to advertise the list more so that other Aussie bloggers can be in the measure. Is there really so few of us that want to know about rankings, or are some Aussie bloggers playing out our national sport of apathy and couldn’t be bothered being included in the measure??
Meg, you need a post grad student to take this on as a thesis! You also need a medal for having kept this list up – the work load must be heavy and all for no remuneration! But….we love you Meg and that’s got to be worth something
Hmm, I ratified myself just out of curiosity a few days ago. I was trying to figure out how to get the widget but although I received confirmation that I’d been processed, my blog isn’t listed. I thought maybe that the only listed the top 100, but there are listings of blogs lower than that. Do you know how it works? I’m not very techie, but not entirely a nuf-nuf. But their pages don’t have a FAQ for me to figure it out.
Apart from that, I’ve been trying to get my Technorati indexed properly the entire time I’ve been on it. They fix it and then it’s gone again. Now it’s 30 days out of date. Urrgh.
Sorry, I’m having a whine… What a wonderful guest I am!
Hi, Meg. You’ve been doing so well in keeping up the Top 100 List, and continuing to find ways to improve it. Thanks for all the work you put in to!
And yes, I agree that certain types of lists and measurements of success DO need some kind of number crunching. It’s like school. Grades may be a good way to show someone’s intelligence/aptitude, but a lot of people ‘slip’ through the system. And, there’s just no way to say how well they’re doing if they’re not very ‘academic’. And yet, there are many undiscovered talents, geniuses and great students who don’t get any medals or scholarship awards.
It’s the same in the world of weblogs. Subscribers, pageviews, comments, citations, PageRanks, etc can help in determining someone’s ‘success’ in blogging. It’s great to see what the numbers say too. But yes, I believe that there’s more to it than just numbers. Many untold stories that the numbers can’t tell.
Yes, unfortunately, there’s no way to automate those things that constitute the ‘more’ part. I guess that’s why I decided to start the Blog Star Awards. I want to help in celebrating those blogging successes that can’t be easily measured by the numbers. Or by popularity voting and such. It’ll be a challenge, but I think it’ll be interesting to see how it works. Like you, I’m sure I will learn a lot along the way. Thanks for the link-up and for quoting me on that.
Looking forward to seeing how you decide to develop the Top 100 List.
The real question is what is blogging success?
I judge it by:
1. the number of people who comment or send me emails thanking me for useful information on my sites that saved them time/money etc
2. Income earnt from the sites
Neither of these can be measured by any external systems
I say stick with what you’re doing now, any other data points eg: pagerank etc are no more or less accurate, they just introduce more variables into the system
Thanks for all your wonderful feedback. Ironically, I’ve been off number crunching the list for this fortnight’s update (in hindsight probably an ill-timed post)!
@ Jen – I too am not sure I deserve to be where I am on the list! I think I have an (unfair) advantage in that 1) I’m hosting the list and 2) am therefore more likely to get visits from people with a keen interest in rankings and more likely to have the Alexa toolbar installed. I actually excluded myself from the list for a while, for this very reason.
Networking is an important part of building up a blog – you can’t rely on people knowing that your blog is there. Being proactive is necessary.
Feedburner does go up and down a bit, but for the most part the big dips are “global”. I’ve noticed that getting a large amount of traffic (from StumbleUpon, Digg (not me) or Google) can cause a spike, which generally normalises over a few days (unfortunately).
@ Leigh – Lol 😀 Yes, time flies hey? Thanks for your kind words. I know we’ve had many discussions about this, and it would be great if someone could come up with a workable solution.
@ Kevin – well done with your model! I love numbers, but that level of computation scares me (I dropped 3 Unit (advanced) maths).
You do raise a very pertinent example with your own blog, which mirrors my concerns – so thank you for sharing that. 850 subscribers is a very commendable number and deserving of a position on a list such as this, but as you suggest your other statistics would probably (erroneously) exclude you.
Many thanks for your wishes.
@ Simon – yes, it’s choked!
But in your definition of “engagement” – how do we truly measure how many people “refer” to it. When you have people commenting on blogs with the sole purpose of getting a link, when you have people producing a meme, a theme, or plugin, or joining a blogroll, are they more or less worthy when counting “citations”.
And then when you look at who depends on what, I’m sure each person’s “must have” list is completely different. To one person it might be a cooking, fashion, celebrity, gardening, motoring or cricket blog, but to you it’s Seth Goodin and John Battelle – so in that sense it’s very subjective. But to get a measure, you just have to quantify the “you” for each blog, which is hard to do.
@ Megan – oh I hope it printed ok! I’ll be interested to hear what the bean counter has to say. In so many ways you too are lucky you’re so different 😉 I’ve got a post about “reach” waiting in draft – it certainly is a measure. A PHD in Blog Analytics – there has to be a call for that, surely? Maybe I could get a grant (is that a stipend?) and be paid to look into it 😉
Thanks for your kind words – and that does count for a lot 😀
@ Cerebralmum – can I have a look into that tomorrow? I’ll respond by email.
@ Shai – thanks 😀 You present a very good analogy. It’s the qualitative measures that can’t be quantified (I know that’s an oxymoron or something!) that often produce a blog star. Good luck with the awards and I’ll be linking to it explicitly in my fortnightly round up.
@ Neerav – it’s good for you to have measures for your own success – obviously they vary from one person to the next. You are very right that they can’t be measured externally.
I have to say I think Google has made page rank completely irrelevant now. I currently have a 0.
Not to mention the fact that they won’t tell anyone how it works, when it will be updated, and they left so many people without a ranking for so long (over 5 months) recently. I think if one is going to use something as a measuring tool, the following needs to apply –
1. one needs to know how it works
2. one needs to know that it is fair to all
3. one needs to know that nobody is penalized or has their ranks lowered deliberately (as seems to be the case with Pay Per Posties and Text Link Ads users and Google Rage Rank)
4. one needs to know how often it is updated
5. one needs to know that everyone is updated at the same time
So based on the above, PR is out, in my opinion.
Cheers,
Snoskred
Say What? At the end of October you were given a PR4, then a few days ago lowered to a 3, and now a zero? HUH?
Who knows what the heck is going on over at the Googleplex, Meg.
I believe I am not the only Australian blogger affected.
This morning I have been in contact with the ACCC. I recommend any Australian blogger affected should do the same. We do have laws here in Australia – the trade practices act is the one I suspect Google is breaking. The ACCC is already taking Google to court over sponsored results.
Cheers,
Snoskred
Hands up, go me too. And it’s a site wide 0. Where as before different sections had different ranks and it depended if there were paid links. Now it’s just 0 everywhere.
Google must still be OK with cats selling links though *hehe*
Leigh
Sorry to hear that mate. Any chance it might be a “pre-adjustment” adjustment?
They can’t be mean to cats – the RSPCA might get called in 😉
I’m not sure how it can be a pre-adjustment adjustment seeing as they adjusted it such a short amount of time ago. One blogger claims to have called Google and been told it was temporary and it would be adjusted back tonight. I don’t know how this is possible – I tried to call them and you can’t seem to speak to a human.
If you’re in Australia and you have been affected, you can call the ACCC. The phone number is 1300 302 502 (Australian callers) or + 612 6243 1305 (overseas callers). The more of us who call, the more chance something might be done about it.
We have choices and my choice is now to look for options other than Google to do certain tasks – and encourage others to do the same. They seem to think they are Big Brother. It is time we took our power back.
I’m writing a post on the topic now, with other options you can use instead of Google.
Cheers,
Snoskred
I’m not a Google apologist by any means (see my post We are the Google Borg. Resistance is futile. You will be Assimilated)
By all means quit using Google services if you want but please don’t bother the poor old ACCC with complaints about pagerank
As a private company Google can measure websites in any way they please, the ACCC has no jurisdiction over pagerank or any other method Google uses to determine search engine rankings
I think that the weighting of Alexa is too high for the reasons that you mention. I noticed that the UK top 100 list took a percentage of Alexa ranking to include in their overall ranking. Most of the other things make sense. I agree with your comments on Ratified. Google Page Rank and Feedburner make up a big part of the ranking.
I’m finally just catching up on your blog posts this week (have been slack!) but I wish there was an easier way to monitor all of these stats & techy mumbo jumbo.
I’ll confess – I haven’t the faintest what any of it means. I have one, for the purposes of seeing if my blog hits a top list one day, but it doesn’t mean much to me. I also want to know my my old pre-domain website still has a higher Technorati rating than my old one, even though I’d say 95% of my readers changed over their links with me.
I’m easily confused. Ah well!
[…] presents Measuring Blog Success and Formulas for Top Lists posted at Dipping into the Blogpond, saying, “There are many measures we can use to gauge a […]
[…] skew Alexa shows towards websites of a technical nature has often frustrated me, and I agree with Darren in hoping that these changes will address […]
[…] have previously discussed the difficulty in determining the rankings and lamented that we don’t live in a perfect world – which would […]