Hard at work

I take requests, too (sometimes)

Yes folks, I take requests. Thank you very much. I'll be here all week and don't forget to tip your waitresses.

retrofire thought it would be interesting to see superimposed graphs of Bush and Blair approval ratings, so I threw one together. The data are taken from the links provided in that post.

Full disclosure on assumptions made in the graph, since "news" organizations don't typically give this: the line for Blair (the lower, green one) is much smoother than Bush's line simply because I didn't have enough data. Also, note that the approval swings appear greater than they actually are because if the graph was fully from 0% to 100%, the lines would be smoother and you'd lose detail. Also, if the poll was taken over several days, I simply took the last date. You should also check back to the original sources to understand how the approval questions were phrased. I took a bit of liberty with them to make them fit together, but I don't think the graph is too far off.

Photobucket - Video and Image Hosting

If anyone can point me to quality, longer-term data for these two, I'd be happy to see if I can put it together similar to this chart.

Update: you might notice that the graph looks different. That's because someone spotted a bug in the date handling. I've fixed the bug and updated the graph.

It's definitely a hack, so don't yell at me :) I embedded the data directly in the program so that you wouldn't need to download separate data files. I should have used some Web modules to pull the data directly, but I was lazy.

If you know Perl and want to replicate this on a Mac, here are some great instructions on how to compile GD for it.

#!/usr/bin/perl

use strict;
use warnings;
use DateTime;
use GD::Graph::lines;
use List::Util 'sum';
use Data::Dumper::Simple;

my @bush_fields = qw(source date approve disapprove unsure delta);
my @bush        = get_records( bush_data(), \@bush_fields );

my ( $bush_dates, $bush_approval ) = get_results( \@bush );
my %bush;
@bush{@$bush_dates} = @$bush_approval;

my @blair_fields = qw(source date approve disapprove delta);
my @blair = get_records( blair_data(), \@blair_fields, 1 );
my ( $blair_dates, $blair_approval ) = get_results( \@blair );
my %blair;
@blair{@$blair_dates} = @$blair_approval;

# synchronize dates
foreach my $key ( keys %blair ) {
    $bush{$key} ||= undef;
}
foreach my $key ( keys %bush ) {
    $blair{$key} ||= undef;
}

my @combined_keys = sort keys %blair;
my @bush_data     = @bush{@combined_keys};
my @blair_data    = @blair{@combined_keys};

my @data = (
    [@combined_keys],
    [@bush_data],
    [@blair_data],
);

my $my_graph = new GD::Graph::lines();

$my_graph->set(
    x_label           => 'Date',
    y_label           => '% Approve',
    title             => 'Approval ratings (Bush, Red.  Blair, Green)',
    y_max_value       => 60,
    y_min_value       => 20,
    y_tick_number     => 8,
    y_label_skip      => 2,
    x_label_skip      => 4,
    box_axis          => 0,
    line_width        => 3,
    x_labels_vertical => 1,
);

$my_graph->plot( \@data );
save_chart( $my_graph, 'bush_blair' );

sub get_records {
    my ( $data, $fields, $british_dates ) = @_;

    my @records;
    my @data_records = split /\n/ => $data;
    foreach (@data_records) {
        my @record = split /\s+/, $_;
        unless ( @$fields == @record ) {
            die "Cannot reconcile line $.: " . Dumper(@record);
        }
        my %record;
        @record{@$fields} = @record;
        $record{date} = get_date( $record{date}, $british_dates );
        push @records => \%record;
    }
    return @records;
}

sub get_date {
    my ( $date, $british_dates ) = @_;
    my ( $month, $day, $year ) = $date =~ m{^(\d+).*?(\d+)/(\d+)$};
    unless ( $month && $day && $year ) {
        die "Line $., date ($date) failed:\n "
          . Dumper( $month, $day, $year );
    }
    if ($british_dates) {

        # Take that, C programmers!
        ( $month, $day ) = ( $day, $month );
    }
    $year = "20$year";    # 'cuz of the format, see ...
    return DateTime->new(
        month => $month,
        day   => $day,
        year  => $year,
    );
}

sub get_results {
    my $records = shift;
    my %results;
    foreach my $record (@$records) {
        my $date = $record->{date};
        my $key  = $date->year . '/'
          . sprintf( "%02d", $date->month ) . '/'
          . sprintf( "%02d", $date->day );
        $results{$key} ||= [];
        push @{ $results{$key} } => $record->{approve};
    }
    my @results = map { [ $_ => average( $results{$_} ) ] }
      sort { $a cmp $b } keys %results;
    my @dates   = map { $_->[0] } @results;
    my @approve = map { $_->[1] } @results;
    return ( \@dates, \@approve );
}

sub average {
    my $list  = shift;
    my $count = @$list;
    return sum(@$list) / $count;
}

sub save_chart {
    my $chart = shift or die "Need a chart!";
    my $name  = shift or die "Need a name!";
    local (*OUT);

    my $ext = 'png'; #$chart->export_format;

    open( OUT, ">", "$name.$ext" )
      or die "Cannot open $name.$ext for write: $!";
    binmode OUT;
    print OUT $chart->gd->$ext;
    close OUT;
}

sub bush_data {
    return <<'END_DATA';
FOX/Opinion_Dynamics_RV      8/8-9/06       36      56      8       -20
ABC/Washington_Post   8/3-6/06    40  58  2   -18
CNN   8/2-3/06    40  59  2   -19
L.A._Times/Bloomberg  7/28-8/1/06   40  58  2   -18
Cook/RT_Strategies_RV     7/28-30/06  39  51  10  -12
USA_Today/Gallup  7/28-30/06  40  56  4   -16
CBS/New_York_Times    7/21-25/06  36  55  9   -19
NBC/Wall_Street_Journal   7/21-24/06  39  56  5   -17
USA_Today/Gallup  7/21-23/06  37  59  4   -22
Diageo/Hotline_RV     7/20-23/06  38  59  3   -21
Pew   7/6-19/06   36  57  7   -21
FOX/Opinion_Dynamics_RV   7/11-12/06  36  53  11  -17
AP-Ipsos_*    7/10-12/06  36  63 ?     -27
Gallup    7/6-9/06    40  55  5   -15
Time  6/27-29/06  35  59  6   -24
FOX/Opinion_Dynamics_RV   6/27-28/06  41  50  9   -9
L.A._Times/Bloomberg  6/24-27/06  41  56  3   -15
USA_Today/Gallup  6/23-25/06  37  60  3   -23
ABC/Washington_Post   6/22-25/06  38  60  2   -22
Diageo/Hotline_RV     6/21-25/06  41  56  3   -15
Pew   6/14-19/06  36  54  10  -18
CNN   6/14-15/06  37  53  10  -16
FOX/Opinion_Dynamics_RV   6/13-14/06  40  52  8   -12
NBC/Wall_Street_Journal   6/9-12/06   37  58  5   -21
CBS   6/10-11/06  33  60  7   -27
USA_Today/Gallup  6/9-11/06   38  56  6   -18
AP-Ipsos_*    6/5-7/06    35  63  ?   -28
Cook/RT_Strategies_RV     6/1-4/06    37  57  6   -20
USA_Today/Gallup  6/1-4/06    36  57  6   -21
Quinnipiac_RV     5/23-30/06  35  58  7   -23
Pew   4/27-5/22/06  33  56  11  -23
Diageo/Hotline_RV     5/18-21/06  37  61  3   -24
FOX/Opinion_Dynamics_RV   5/16-18/06  35  56  8   -21
CBS   5/16-17/06  35  60  5   -25
CNN   5/16-17/06  36  57  7   -21
ABC/Washington_Post   5/11-15/06  33  65  2   -32
Newsweek  5/11-12/06  35  59  6   -24
Gallup    5/8-11/06   33  61  6   -28
CBS/New_York_Times    5/4-8/06    31  63  6   -32
CNN   5/5-7/06    34  58  8   -24
USA_Today/Gallup  5/5-7/06    31  65  5   -34
FOX/Opinion_Dynamics_RV   5/2-3/06    38  53  9   -15
AP-Ipsos_*    5/1-3/06    33  65  ?   -32
CBS   4/28-30/06  33  58  9   -25
USA_Today/Gallup  4/28-30/06  34  63  3   -29
Cook/RT_Strategies    4/27-30/06  36  59  5   -23
NBC/Wall_Street_Journal   4/21-24/06  36  57  7   -21
CNN   4/21-23/06  32  60  8   -28
FOX/Opinion_Dynamics_RV   4/18-19/06  33  57  10  -24
Pew   4/7-16/06   35  55  10  -20
Gallup    4/10-13/06  36  59  5   -23
L.A._Times/Bloomberg  4/8-11/06   39  57  4   -18
USA_Today/Gallup  4/7-9/06    37  60  3   -23
ABC/Washington_Post   4/6-9/06    38  60  2   -22
CBS   4/6-9/06    37  56  7   -19
Cook/RT_Strategies    4/6-9/06    37  56  7   -19
FOX/Opinion_Dynamics_RV   4/4-5/06    36  53  10  -17
AP-Ipsos_*    4/3-5/06    36  62  ?   -26
Time  3/29-30/06  37  57  6   -20
Time  3/22-23/06  39  56  6   -17
Newsweek  3/16-17/06  36  58  6   -22
Gallup    3/13-16/06  37  59  5   -22
FOX/Opinion_Dynamics_RV   3/14-15/06  39  51  11  -12
NPR_LV    3/12-14/06  39  58  3   -19
NBC/Wall_Street_Journal   3/10-13/06  37  58  5   -21
CNN/USA_Today/Gallup  3/10-12/06  36  60  4   -24
CBS   3/9-12/06   34  57  9   -23
Pew   3/8-12/06   33  57  10  -24
AP-Ipsos_*    3/6-8/06    37  60   ?  -23
ABC/Washington_Post   3/2-5/06    41  58  1   -17
FOX/Opinion_Dynamics_RV   2/28-3/1/06   39  54  7   -15
CNN/USA_Today/Gallup  2/28-3/1/06   38  60  2   -22
L.A._Times/Bloomberg  2/25-3/1/06   38  58  4   -20
Quinnipiac_RV     2/21-28/06  36  58  6   -22
Cook/RT_Strategies    2/23-26/06  40  54  6   -14
CBS   2/22-26/06  34  59  7   -25
Diageo/Hotline_RV     2/16-19/06  45  52  2   -7
Time  2/15-16/06  40  54  5   -14
WNBC/Marist_RV    2/13-15/06  40  57  3   -17
CNN/USA_Today/Gallup  2/9-12/06   39  56  4   -17
Gallup    2/6-9/06    42  55  4   -13
FOX/Opinion_Dynamics_RV   2/7-8/06    44  47  9   -3
AP-Ipsos_*    2/6-8/06    40  57  ?   -17
Pew   2/1-5/06    40  52  8   -12
NBC/Wall_Street_Journal   1/26-29/06  39  54  7   -15
Time  1/24-26/06  41  55  4   -14
ABC/Washington_Post   1/23-26/06  42  56  2   -14
FOX/Opinion_Dynamics_RV   1/24-25/06  41  51  8   -10
Cook/RT_Strategies    1/22-25/06  47  50  3   -3
L.A._Times/Bloomberg  1/22-25/06  43  54  3   -11
CBS/New_York_Times    1/20-25/06  42  51  7   -9
CNN/USA_Today/Gallup  1/20-22/06  43  54  4   -11
Diageo/Hotline_RV     1/12-15/06  46  53  2   -7
Gallup    1/9-12/06   43  53  4   -10
FOX/Opinion_Dynamics_RV   1/10-11/06  42  49  9   -7
CNN/USA_Today/Gallup  1/6-8/06    43  54  3   -11
ABC/Washington_Post   1/5-8/06    46  52  2   -6
CBS   1/5-8/06    41  52  7   -11
Pew   1/4-8/06    38  54  8   -16
AP-Ipsos_*    1/3-5/06    40  59  ?   -19
Gallup    12/19-22/05     43  53  4   -10
CNN/USA_Today/Gallup  12/16-18/05     41  56  3   -15
ABC/Washington_Post   12/15-18/05     47  52  1   -5
NPR_LV    12/15,17-18/05     44  54  2   -10
FOX/Opinion_Dynamics_RV   12/13-14/05     42  51  7   -9
Diageo/Hotline_RV     12/12-13/05     50  47  3   +3
NBC/Wall_Street_Journal   12/9-12/05  39  55  6   -16
CNN/USA_Today/Gallup  12/9-11/05  42  55  3   -13
Cook/RT_Strategies    12/8-11/05  42  55  2   -13
Pew   12/7-11/05  38  54  8   -16
Gallup    12/5-8/05   43  52  5   -9
AP-Ipsos_*    12/5-7/05   42  57   ?  -15
CBS/New_York_Times    12/2-6/05   40  53  7   -13
Quinnipiac_RV     11/28-12/4/05     40  54  7   -14
Time  11/29-12/1/05     41  53  5   -12
FOX/Opinion_Dynamics_RV   11/29-30/05     42  48  10  -6
Cook/RT_Strategies    11/17-20/05     41  52  7   -11
Gallup    11/17-20/05     38  57  5   -19
Diageo/Hotline_RV     11/11-15/05     39  59  2   -20
CNN/USA_Today/Gallup  11/11-13/05     37  60  3   -23
Newsweek  11/10-11/05     36  58  6   -22
Gallup    11/7-10/05  40  55  5   -15
FOX/Opinion_Dynamics_RV   11/8-9/05   36  53  11  -17
AP-Ipsos_*    11/7-9/05   37  61  ?   -24
NBC/Wall_Street_Journal   11/4-7/05   38  57  5   -19
Pew   11/3-6/05   36  55  9   -19
AP-Ipsos_*    10/31-11/2/05     37  59  ?   -22
ABC/Washington_Post   10/30-11/2/05     39  60  1   -21
CBS   10/30-11/1/05     35  57  8   -22
CNN/USA_Today/Gallup  10/28-30/05     41  56  3   -15
ABC/Washington_Post   10/28-29/05     39  58  3   -19
FOX/Opinion_Dynamics_RV   10/25-26/05     41  51  8   -10
Gallup    10/24-26/05     41  56  3   -15
Pew   10/12-24/05     40  52  8   -12
CNN/USA_Today/Gallup  10/21-23/05     42  55  3   -13
WNBC/Marist_RV    10/12-13&17/05    41  53  6   -12
CNN/USA_Today/Gallup  10/13-16/05     39  58  3   -19
Diageo/Hotline_RV     10/12-16/05     40  57  2   -17
FOX/Opinion_Dynamics_RV   10/11-12/05     40  51  9   -11
NBC/Wall_Street_Journal   10/8-10/05  39  54  7   -15
Pew   10/6-10/05  38  56  6   -18
AP-Ipsos_*    10/3-5/05   39  58   ?  -19
CBS   10/3-5/05   37  58  5   -21
Newsweek  9/29-30/05  40  53  7   -13
FOX/Opinion_Dynamics_RV   9/27-28/05  45  47  8   -2
CNN/USA_Today/Gallup  9/26-28/05  45  50  5   -5
ABC/Washington_Post   9/8-11/05   42  57  1   -15
Pew   9/8-11/05   40  52  8   -12
Time  8/7-8/05    42  52  6   -10
END_DATA
}

sub blair_data {
    return <<'END_DATA';
YouGov/Telegraph    26/07/06    27      63      -36
Ipsos-MORI/FT   24/07/06    23  67  -44
Ipsos-MORI  26/06/06    32  60  -28
Ipsos-MORI/Sun  30/05/06    26  67  -41
Ipsos-MORI/FT   02/05/06    29  64  -35
YouGov/Sunday_Times     29/04/06    33  64  -31
YouGov/Telegraph    29/03/06    30  60  -30
Ipsos-MORI  21/03/06    31  62  -31
YouGov/Sunday_Times     17/03/06    36  61  -25
ICM/Guardian    12/03/06    37  45  -8
Ipsos-MORI/FT   23/01/06    36  57  -21
YouGov/Telegraph    26/01/06    33  57  -24
Ipsos-MORI/Observer     22/11/05    37  55  -18
MORI/Observer   25/10/05    36  55  -19
MORI    26/09/05    31  60  -29
ICM/Guardian    24/09/05    39  58  -19
MORI    15/08/05    39  51  -12
ICM/Guardian    14/08/05    47  45  +2
MORI/FT     18/07/05    44  47  -3
YouGov/Telegraph    08/07/05    49  42  +7
MORI    20/06/05    39  52  -13
MORI/FT     23/05/05    39  55  -16
YouGov/Sunday_Times     09/04/05    46  50  -4
MORI/FT     22/03/05    34  59  -25
YouGov/Sunday_Times     19/03/05    46  50  -4
MORI/FT     21/02/05    35  57  -22
MORI/Observer   24/01/05    33  57  -24
ICM/Guardian    23/01/05    38  50  -12
YouGov/Sunday_Times     22/01/05    40  56  -16
END_DATA
}

  • Current Mood: geeky geeky
Glorious! How I love charts.

The journey the lines make varies from opposition to a somewhat parallel course - interesting how sometimes Tony's highs are Bush's lows. The peaks and valleys may be related - one theory would be that Bush does something terrible, the British people recoil in disgust and Tony makes a damage control speech. One thing that is not represented on the chart but clearly exerts a strong influence is that the US has taken control of the news and it's purpose is less to inform than it is to control public opinion.

That being said, my next step will be assigning key events around dramatic swings to see what these charts reveal about how we are being abused. I wish I could take the day off work to do it, but will get to it in the next day or two. Here are some sources I will use - but will have to search to see if someone did the job already.
Events related to Tony Blair
http://www.harpers.org/TonyBlair.html#44a997c4e9577
Events related to George Bush
http://www.harpers.org/GeorgeWBush.html#44a997c4e9577
News of the week in review
http://www.harpers.org/WeeklyReview.html
Fun!
Thanks :)
I did start to do it and it is very frustrating. I was unable to find a clear source of highlights for both US and UK news. And I can't find a news source that will gave me headlines from a specific date.
So after a bad start I am sneaking away quietly. Please accept my apologies. But I will leave you with the following -

09-07-05 In the aftermath of Thursday's bombings, Mr Blair's approval rating has flipped from negative to positive for the first time in five years.