A poor substitute for Mega-Critic & a call for a better one
As we reach the end of August, I thought I'd try my wholly inadequate hand at filling the void left by the absence of Mega-Critic. As usual, the following list is based on the rawest, most rudimentary "calculation" imaginable - Metascore + number of Metacritic reviews. That's it. If anyone's been working on anything more complex, please post it. In the meantime, here are the top 50 albums of 2011, as of August 31:
ARTIST / TITLE / METASCORE / REVIEWS / TOTAL
1 P J Harvey / Let England Shake / 86 / 42 / 128
2 Bon Iver / Bon Iver / 86 / 41 / 127
3 Fleet Foxes / Helplessness Blues / 85 / 42 / 127
4 Beastie Boys / Hot Sauce Committee, Pt. 2 / 83 / 42 / 125
5 Destroyer / Kaputt / 84 / 37 / 121
6 TV on the Radio / Nine Types of Light / 82 / 39 / 121
7 tune-Yards / Who-Kill / 86 / 34 / 120
8 Fucked Up / David Comes to Life / 86 / 33 / 119
9 James Blake / James Blake / 81 / 38 / 119
9 Stephen Malkmus & The Jicks / Mirror Traffic / 81 / 38 / 119
11 Radiohead / The King of Limbs / 80 / 39 / 119
12 Lykke Li / Wounded Rhymes / 84 / 33 / 117
13 Shabazz Palaces / Black Up / 82 / 34 / 116
14 Yuck / Yuck / 81 / 34 / 115
15 Battles / Gloss Drop / 79 / 36 / 115
16 Foo Fighters / Wasting Light / 78 / 37 / 115
17 The Decemberists / The King is Dead / 77 / 38 / 115
18 Iron & Wine / Kiss Each Other Clean / 77 / 37 / 114
18 Panda Bear / Tomboy / 77 / 37 / 114
20 Jay-Z & Kanye West / Watch the Throne / 75 / 39 / 114
21 Okkervil River / I Am Very Far / 80 / 33 / 113
22 Paul Simon / So Beautiful or So What / 85 / 27 / 112
22 Wild Beasts / Smother / 85 / 27 / 112
24 My Morning Jacket / Circuital / 77 / 35 / 112
25 The Strokes / Angles / 71 / 41 / 112
26 Thurston Moore / Demolished Thoughts / 78 / 33 / 111
27 Mogwai / Hardcore Will Never Die, But You Will / 77 / 34 / 111
28 The Horrors / Skying / 83 / 27 / 110
29 Kurt Vile / Smoke Ring for My Halo / 82 / 28 / 110
30 Smith Westerns / Dye It Blonde / 81 / 29 / 110
30 The War on Drugs / Slave Ambient / 81 / 29 / 110
32 Adele / 21 / 76 / 34 / 110
33 Gang Gang Dance / Eye Contact / 83 / 26 / 109
34 Gil Scott-Heron / Jamie XX / We're New Here / 82 / 27 / 109
35 Beirut / The Rip Tide / 79 / 30 / 109
36 Deerhoof / Deerhoof Vs. Evil / 77 / 32 / 109
36 The Mountain Goats / All Eternals Deck / 77 / 32 / 109
38 Elbow / Build a Rocket Boys! / 83 / 25 / 108
38 Gillian Welch / The Harrow and the Harvest / 83 / 25 / 108
40 The Antlers / Burst Apart / 81 / 27 / 108
41 The Kills / Blood Pressures / 76 / 32 / 108
42 R.E.M. / Collapse Into Now / 71 / 37 / 108
43 Low / C'Mon / 80 / 27 / 107
44 Toro y Moi / Underneath the Pine / 79 / 28 / 107
45 Junior Boys / It's All True / 78 / 29 / 107
46 Explosions in the Sky / Take Care, Take Care, Take Care / 77 / 30 / 107
47 Tyler, the Creator / Goblin / 72 / 35 / 107
47 Death Cab for Cutie / Codes and Keys / 72 / 35 / 107
49 J. Mascis / Several Shades of Why / 78 / 28 / 106
50 The Low Anthem / Smart Flesh / 76 / 30 / 106
NOTE: However anyone calculates the most acclaimed albums of 2011 thus far, it's all likely going to be blown to pieces come Tuesday, September 13 with the release of Das Racist, Girls, Laura Marling, St. Vincent and Wild Flag, among others. What a day!
Re: A poor substitute for Mega-Critic & a call for a better one
My offer to set up a program where people can log in and add reviews to, then have it do a win/loss based calculation similar to Henrik's AM formula, is open if somebody else provides the server space.
Re: A poor substitute for Mega-Critic & a call for a better one
Zorg
Quite sad how much Bill Callahan's been forgotten by the critics. I love his album.
This is by far the biggest discrepancy between Metacritic (which Stefan used as a base) and Mega-Critic (which incorporated European sources not covered by the site). Callahan must be an even bigger cult artist overseas than he is in the U.S. and UK - Apocalypse has a Metascore of 75, with only 25 reviews, whereas it was all the way up at #7 on the last Mega-Critic list. Most of the top albums of the year shake out the same no matter what combination of sources you use, but this one is the proverbial sore thumb.
Re: A poor substitute for Mega-Critic & a call for a better one
I've redone Harold's analysis using a Bayesian approach. The idea here is that the more reviews an album has, the closer its metascore is to the true value - what its metascore would be if every music critic in the world had reviewed it. IMDB does the same thing when ranking movies if you're wondering where you've seen this before.
More precisely I used the following formula to rank the albums:
score = [(Album's Raw Score) * (# of reviews for this album) + (Average raw score) * (average # of reviews)]/([(# of reviews for this album) + (average $ of reviews)] of .
Average Metascore = 79.8, Average # of Reviews = 33.24 (among the 50 that were listed here)
Although it may be too late for the best of the 2000s poll. I think this formula should be considered for the final round of future polls. It will eliminated the bias against less well known albums in less arbitrary way. If every voter votes for an album than its weighted score is just the average its raw score. If no one votes for an album its weighted score is just the average of the every albums raw score.
Rank / Artist / Album / Metascore / # of Reviews / Weighted Score
1 / PJ Harvey / Let England Shake / 86 / 42 / 83.26
2 / Bon Iver / Bon Iver / 86 / 41 / 83.22
3 / tune-Yards / Who-Kill / 86 / 34 / 82.94
4 / Fucked Up / David Comes to Life / 86 / 33 / 82.89
5 / Fleet Foxes / Helplessness Blues / 85 / 42 / 82.7
6 / Paul Simon / So Beautiful or So What / 85 / 27 / 82.13
7 / Wild Beasts / Smother / 85 / 27 / 82.13
8 / Destroyer / Kaputt / 84 / 37 / 82.01
9 / Lykke Li / Wounded Rhymes / 84 / 33 / 81.89
10 / Beastie Boys / Hot Sauce Committee, Pt. 2 / 83 / 42 / 81.59
11 / The Horrors / Skying / 83 / 27 / 81.23
12 / Gang Gang Dance / Eye Contact / 83 / 26 / 81.2
13 / Elbow / Build a Rocket Boys! / 83 / 25 / 81.17
14 / Gillian Welch / The Harrow and the Harvest / 83 / 25 / 81.17
15 / TV on the Radio / Nine Types of Light / 82 / 39 / 80.99
16 / Shabazz Palaces / Black Up / 82 / 34 / 80.91
17 / Kurt Vile / Smoke Ring for My Halo / 82 / 28 / 80.81
18 / Gil Scott-Heron / Jamie XX / We're New Here / 82 / 27 / 80.79
19 / James Blake / James Blake / 81 / 38 / 80.44
20 / Stephen Malkmus & The Jicks / Mirror Traffic / 81 / 38 / 80.44
21 / Yuck / Yuck / 81 / 34 / 80.41
22 / Smith Westerns / Dye It Blonde / 81 / 29 / 80.36
23 / The War on Drugs / Slave Ambient / 81 / 29 / 80.36
24 / The Antlers / Burst Apart / 81 / 27 / 80.34
25 / Radiohead / The King of Limbs / 80 / 39 / 79.91
26 / Okkervil River / I Am Very Far / 80 / 33 / 79.9
27 / Low / C'Mon / 80 / 27 / 79.89
28 / Toro y Moi / Underneath the Pine / 79 / 28 / 79.43
29 / Beirut / The Rip Tide / 79 / 30 / 79.42
30 / Battles / Gloss Drop / 79 / 36 / 79.38
31 / J. Mascis / Several Shades of Why / 78 / 28 / 78.98
32 / Junior Boys / It's All True / 78 / 29 / 78.96
33 / Thurston Moore / Demolished Thoughts / 78 / 33 / 78.9
34 / Foo Fighters / Wasting Light / 78 / 37 / 78.85
35 / Explosions in the Sky / Take Care, Take Care, Take Care / 77 / 30 / 78.47
36 / Deerhoof / Deerhoof Vs. Evil / 77 / 32 / 78.43
37 / The Mountain Goats / All Eternals Deck / 77 / 32 / 78.43
38 / Mogwai / Hardcore Will Never Die, But You Will / 77 / 34 / 78.38
39 / My Morning Jacket / Circuital / 77 / 35 / 78.36
40 / Iron & Wine / Kiss Each Other Clean / 77 / 37 / 78.33
41 / Panda Bear / Tomboy / 77 / 37 / 78.33
42 / The Decemberists / The King is Dead / 77 / 38 / 78.31
43 / The Low Anthem / Smart Flesh / 76 / 30 / 78
44 / The Kills / Blood Pressures / 76 / 32 / 77.94
45 / Adele / 21 / 76 / 34 / 77.88
46 / Jay-Z & Kanye West / Watch the Throne / 75 / 39 / 77.21
47 / Death Cab for Cutie / Codes and Keys / 72 / 35 / 75.8
48 / Tyler, the Creator / Goblin / 72 / 35 / 75.8
49 / R.E.M. / Collapse Into Now / 71 / 37 / 75.16
50 / The Strokes / Angles / 71 / 41 / 74.94
Re: A poor substitute for Mega-Critic & a call for a better one
Ahh, you're talking about regression toward the mean. I like that approach. It's probably the most fair parametric way to summarize album reviews.
Have you considered using AnyDecentMusic rather than Metacritic? Metacritic weighs albums arbitrarily and spaces the scores out from the mean artificially.
Re: A poor substitute for Mega-Critic & a call for a better one
Chris, Stefan used a Bayesian approach as well, but instead of the average metascore and the average # of reviews he used the midpoint of the possible range and an arbitrary value (50 instead of 79.8 and 10 instead of 33.24), which gave albums with many reviews an advantage, aiming to predict the EOY lists. With your formula, the albums' number of reviews have a minor influence.
Re: A poor substitute for Mega-Critic & a call for a better one
Otisredding
I do not know if anyone is interested in this but I continue .
Otis, I think we're very interested in it - we needed something like this. I'm guessing everyone's waiting until the end to comment. Intriguing results so far - particularly the Girls album, which by my reckoning is only the 4th-most-acclaimed record to be released -yesterday-, showing up at #21.
Re: A poor substitute for Mega-Critic & a call for a better one
Thanks for doing this. Even though it's different from Mega-Critic, the continuity is impressive - the newer entries are slotted in pretty close to where they'd likely be on MC.
Re: A poor substitute for Mega-Critic & a call for a better one
This is great, whoever has been involved with any of this stuff. I'm not big on computer stuff, or algorithms, so I can't help out unfortunately, but I just want whoever's doing all this to know that I'm loving this stuff.
Thanks so much.