Using Metacritic to include albums from the most recent year is of course a good idea, as it is the only way to make sure that even rather recent albums have the chance to compare against the entire history.
Yet, there is a weakness about Metacritic: Albums by obscure acts (or acts that were obscure prior to the release of that album) tend to dominate more than on the yearlists.
Why? Simply because once an act is established, there is always a number of critics who are after that act, so even if most critics give a rave review (meaning the album will show up in most year-end lists) a few negative critics will usually drag the average down, so that the album doesn't get to enter the list until the year-end lists have been released.
But there is nothing to do about this, I guess, unless you actually weigh Metacritic entries by established acts somewhat more than Metacritic entries by less established acts. Still means that the list isn't really "complete" until the year-end lists by various publications are in. For instance, I am pretty sure that Madonna and Depeche Mode will be in the Top 2500 with their 2005 albums once those year-end lists have been released, but by now their Metacritic score is slightly too low.
12 or 14 reviews used for Madonna's albums at the moment, when there's loads more out there. Hopefully more will be added. However, I don't think her score will increase too much. There actually are a few negative reviews out there (NY Newsday's scatching review stood out big time. lol) Whether they'e included, we'll see.
I think some of the numeral ratings applied are odd at times (i.e. the critique sounds like it would have a better rating than what's listed), but I guess there's a method to the madness.
I think an even bigger issue is the number of reviews that factor in. It seems that albums by established acts that aren't "buzzworthy" don't have as many reviews in metacritic. I notice that new albums are included here not only based on the score itself, but also the number of reviews. This tends to overvalue albums by relative unknowns. For instance, while most of the albums in the current list here have 25-35 reviews over at metacritic, Stevie Wonder's album has only received 13 reviews. I'm not saying he would be included anyway with a metascore of only 61, but even if his metascore was a lofty 90, he wouldn't make the cut. Other decently acclaimed albums that suffer from not enough reviews seem to be Madonna, Fiona Apple and Kate Bush.
I weigh in both the score and number of reviews to get what I think is a ranking of acclaimed albums that is as fair as possible. If "old" and "established" artists have few reviews at metacritic, this could be because many of the magazines that the metacritic staff have included are indie magazines. However, based on a quick look at the end of year lists that have been found so far, I don't think the current Acclaimed Music ranking for 2005 is too bad. (Although you could argue that there are too many zines that do not look beyond their favorite indie bands!)
Metacritic isn't too bad, but you will at least probably have Coldplay and Madonna (possibly Depeche Mode) entering the Top 2500 based on various critics list. They score well at Metacritic as well, but now quite well enough to hit the Top 2500 based on Metacritic alone.
I'm not saying that my methods are perfect, but of the bands/artists you mentioned, only Coldplay is in Stefan's top 50, at place 38. So they are not doing too well in the year-end lists either.
