AEMO case study combined
In [1]:
!pip install --upgrade python-highcharts
from IPython.core.display import clear_output
from time import sleep
sleep(2)
clear_output()
In [2]:
import csv
from datetime import datetime
from highcharts import Highchart, Highstock
from IPython.core.display import display, HTML
In [3]:
# Set the default options that we will use for every chart
default_chart_options = {
    'chart': {
        'zoomType': 'x'
    },
    'title': {
        'text': None,
        'style': {
            'fontWeight': 'bold',
            'color': '#616161',
            'fontSize': 20
        }
    },
    'yAxis': {
        'opposite': False,
        'title': {
            'style': {
                'fontWeight': 'bold',
                'color': '#000000',
                'fontSize': 15
            }
        }
    },
    'xAxis': {
        'title': {
            'style': {
                'fontWeight': 'bold',
                'color': '#000000',
                'fontSize': 15
            }
        }
    },
    'tooltip': {
        'pointFormat': '<b>{point.y:.2f}</b>,  {series.name}<br/>'
    },
    'legend': {
        'enabled': True
    },
    'rangeSelector': {
        'enabled': False
    },
    'navigator': {
        'enabled': False
    },
    'scrollbar': {
        'enabled': False
    },
    'lang': {
        'numericSymbols': [ "k" , "M" , "B" , "T" , "Qd" , "Qn"]
    }
}
In [4]:
####################################### This is the general setting of table ######################################
In [5]:
display(HTML("""
<style>
.rendered_html tbody tr:nth-child(odd) {
    background: ##f5f5f5; }
.rendered_html tr, .rendered_html th, .rendered_html td, .rendered_html table {
    text-align: right;
    vertical-align: middle;
    padding: 0.5em 0.5em;
    line-height: normal;
    white-space: normal;
    max-width: none;
    border: none;
}
</style>
"""))
In [6]:
def read_csv(filename, date_column, date_format='%Y-%m-%d'):
    series = {}
    with open(filename, 'r') as fh:
        reader = csv.DictReader(fh, delimiter=',')
        for row in reader:
            for column in row:
                if column != date_column:
                    if column not in series:
                        series[column] = []
                    if row[column]:
                        series[column].append([datetime.strptime(row[date_column], date_format), float(row[column])])
    for serie in series:
        series[serie].sort(key=lambda x: x[0])
    return series
In [7]:
grouped_bar_line_chart_options = {
    # custom
    'title': {
        'text': 'Regional Reference Price'
    },
    'xAxis': {
            'title': {
                'text': None
            }
    },
    'yAxis': [
        {
            'title': {
                'text': 'Megawatt'
            },
            'offset': 30,
            'opposite': False
        },
        {
            'title': {
                'text': 'Peak price'
            }
        },
    ],
    'legend': {
        'reversed': True
    },
    'plotOptions': {
        'series': {
            'dataGrouping': {
                'groupPixelWidth': 40,
            }
        }
    },
}

Executive Summary

This report examines changes to the health/quality of electricity markets around 3 major events, (i) the withdrawal of Hazelwood power station, (ii) AEMC’s rebidding market rule change, and (iii) the heatwave period in NSW around February 2017.

The results suggest that Hazelwood’s closure has reduced generation capacity in Victoria which may have impacted the region’s wholesale electricity prices. The withdrawal also reduced surplus capacity for export to other states and suggests that NSW may be increasingly reliant on Queensland for electricity imports. There are some potential observations that total revenue for generators in Victoria has improved ex-post event (including Loy Yang B which is owned by Engie). There appears to be no significant observations around media speculation on Hazelwood’s withdrawal (24 September 2016) and the official announcement (3 November 2016).

Initial findings in this study suggest that after the rebidding rule change, there appears to be less late rebids overall. However, there are observations in Queensland that late rebidding behaviour has shifted from the last to first dispatch interval. Regression analysis suggests that rebidding rule change may have an effect on electricity prices changes across settlement periods in Queensland. There are also suggestions that late rebids during multiple (prior) dispatch intervals have an impact on changes in electricity prices. Generally, it is also noted that higher positive price spikes are associated with more late rebids.

Intraday analysis around the NSW heatwave period illustrates a shift in offered capacity from the \$35 - \$9,999 price range to bands below \$0. There are observations that the majority of capacity offered is at the tails (i.e. < \$0 and > \$10,000) with limited capacity offerings in between \$35 - \$9,999. During event days (i.e. where settlement prices exceeds \$10,000), there also appears to be more rebids compared to the relevant control periods$.^1$

The metrics in this study have been incorporated into the MQEnergy platform. There may potentially be opportunities to apply these measures broadly to other events involving (i) a major supply disruption (similar to Hazelwood’s withdrawal), (ii) market design changes (e.g. rebidding rule revision), and (iii) demand shocks (e.g. heatwave) to observe their impacts on NEM and each state individually.

[1]: Due to the length of the results, other tables can be provided upon request.

1. Event - Hazelwood Power Station

This section reports the findings on Hazelwood’s power station withdrawal from NEM. The closure process began on the 27 March 2017 and was completed on 1 April 2017 (inclusive). The pre-event originates before 26 March 2017 and post-event period is from 2 April 2017.

Results suggest this event has an impact on Victoria’s generation capacity which may have increased wholesale electricity prices in the region. This is also likely to have reduced Victoria’s surplus capacity for export to other states. Market concentration (power) has increased and price volatility has decreased, ex-post event.

Prior to Hazelwood’s closure, Victoria’ (but not Queensland) surplus has an impact on electricity prices in NSW. After the withdrawal, NSW appears less reliant on Victoria and more on Queensland. This result is robust (for a matched sample) after controlling for regional/local (a) demand, (b) offered capacity, (c) price volatility and (d) market power.

With the withdrawal of Hazelwood, the proportion of coal’s contribution to the electricity output in Victoria has decreased. Ceteris paribus, the proportion (%) contributed by other sources has increased relative to coal. It is also observed that coal’s contribution towards electricity generation has increased in Queensland.

There are some observations that Victorian generators exhibit statistically significant increases approximately 90-days after Hazelwood’s withdrawal. This includes Hazelwood’s majority owner (Engie) for Loy Yang (owned by subsidiary IPM Australia Limited).

There were no significant observations around media speculation on Hazelwood’s withdrawal (24 September 2016) and the official announcement (3 November 2016).

Key Findings

In [8]:
# MAP
display(HTML('<script src="https://mqd-public-static.s3.amazonaws.com/custom_js/highstockmap.js"></script>'))
In [9]:
def load_aggregated_data(metrics, date_from=None, date_to=None):
    date_columns = ['settlement_date', '']
    data = {}
    for metric in metrics:
        sum_data = {}
        data_counts = {}
        with open(metrics[metric]) as fh:
            reader = csv.DictReader(fh)
            for row in reader:
                skip_row = False
                if date_from is not None or date_to is not None:
                    for date_column in date_columns:
                        if date_column in row:
                            date = datetime.strptime(row[date_column], '%Y-%m-%d')
                            if date_from and date < date_from:
                                skip_row = True
                            if date_to and date > date_to:
                                skip_row = True
                            break
                if skip_row:
                    continue
                for column in row:
                    if column in date_columns:
                        continue
                    if column not in sum_data:
                        sum_data[column] = 0
                        data_counts[column] = 0
                    sum_data[column] += float(row[column])
                    data_counts[column] += 1
        for market in sum_data:
            if market not in data:
                data[market] = {}
            data[market][metric] = sum_data[market] / data_counts[market]
    return data
In [10]:
state_to_region = {
    'NSW1': 'AU.NS',
    'VIC1': 'AU.VI',
    'QLD1': 'AU.QL',
    'TAS1': 'AU.TS',
    'SA1': 'AU.SA',
}

metric1 = 'Daily VWAP'

metric_order = ['Daily VWAP', 'Daily Total Offered Capacity', 'Daily Surplus Capacity']

metrics = {
    'Daily VWAP': 'daily_vwap_2015_01_2017_09.csv',
    'Daily Total Offered Capacity': 'daily_total_offered_capacity_2015_01_2017_09.csv',
    'Daily Surplus Capacity': 'daily_surplus_capacity_2015_01_2017_09.csv',
}

line_chart_type = {
    'Daily VWAP': 'column',
    'Daily Total Offered Capacity': 'line',
    'Daily Surplus Capacity': 'area',
}

data = load_aggregated_data({metric1: metrics[metric1]})
data = [[state_to_region[state], data[state][metric1]] for state in data]

display(HTML(
    '<script type="text/javascript"> '
    'var geomap_data = ' + str(data) + ';'
    '</script>'))
In [11]:
display(HTML("""
<div id="geomap1" style="width:800px; height:600px;"></div>
<script type="text/javascript">

function monitor_page() {
    setTimeout(function() {
        if (!$('#geomap1').hasClass('loaded')) {
            if (typeof Highcharts != 'undefined') {
                $('#geomap1').addClass('loaded');
                load_geomap();
            } else {
                monitor_page();
            }
        }
    }, 1000);
}
monitor_page();

var mapChart = null;

function region_to_state(region) {
    if (region == 'AU.NS') return 'NSW1';
    if (region == 'AU.VI') return 'VIC1';
    if (region == 'AU.QL') return 'QLD1';
    if (region == 'AU.TS') return 'TAS1';
    if (region == 'AU.SA') return 'SA1';
    throw Error('Unknown region ' + region);
}

function load_geomap() {
    
    Highcharts.wrap(Highcharts.Point.prototype, 'select', function (proceed) {
        proceed.apply(this, Array.prototype.slice.call(arguments, 1));
        var points = mapChart.getSelectedPoints();
        var selected_region = points[0].code_hasc;
        $('.geo_subchart').hide();
        $('#geo_' + region_to_state(selected_region) + '_1_subchart').show();
        $('#geo_' + region_to_state(selected_region) + '_2_subchart').show();
        $('#geo_' + region_to_state(selected_region) + '_3_subchart').show();
    });

    $.getJSON('https://www.highcharts.com/samples/data/jsonp.php?filename=australia.geo.json&callback=?', function (geojson) {

        // Initiate the chart
        mapChart = Highcharts.mapChart('geomap1', {
            credits: {
                enabled: false
            },
            title: {
                text: "Impact of Hazelwood's withdrawal"
            },
            mapNavigation: {
                enabled: true,
                buttonOptions: {
                    verticalAlign: 'bottom'
                }
            },
            colorAxis: {
                min: 60,
                type: 'logarithmic',
                //minColor: '#FFFFFF',
                //maxColor: '#FF0000',
                stops: [
                    [0, '#267302'], //GREEN
                    [0.5, '#FFFFFF'], //WHITE
                    [1, '#FF0000'] //RED
                ]
            },
            tooltip: {
                pointFormat: '<span style="color:{point.color}">\u25CF</span><span> {point.name}: <b>{point.value:.2f}</b><br/></span>'
            },
            series: [{
                data: geomap_data,
                mapData: geojson,
                color: 'red',
                joinBy: ['code_hasc', 0],
                keys: ['code_hasc', 'value'],
                name: '""" + metric1 + """',
                allowPointSelect: true,
                cursor: 'pointer',
                states: {
                    select: {
                        color: '#3F5666',
                        borderColor: 'white',
                        dashStyle: 'shortdot'
                    },
                    hover: {
                        color: '#3e7092'
                    }
                },
                dataLabels: {
                    enabled: true,
                    format: '{point.properties.postal}'
                }
            }]
        });
    });
}

</script>
"""))
In [12]:
state_data = {}
metric_count = 0
for metric in metric_order:
    state_data[metric] = read_csv(metrics[metric], 'settlement_date')
    metric_count += 1
    for state in state_data[metric]:
        line_id = state + '_' + str(metric_count)
        state_data[metric][state] = [[(a -  datetime.utcfromtimestamp(0)).total_seconds() * 1000.0, b] for a, b in state_data[metric][state]]
        display(HTML("""
            <div id="geo_""" + line_id + """_subchart" class="geo_subchart" style="width:900px; height:400px;"></div>
            <script type="text/javascript">

            function monitor_page_""" + line_id + """() {
                setTimeout(function() {
                    if (!$('#geo_""" + line_id + """_subchart').hasClass('loaded')) {
                        if (typeof Highcharts != 'undefined') {
                            $('#geo_""" + line_id + """_subchart').addClass('loaded');
                            load_geomap_""" + line_id + """();
                        } else {
                            monitor_page_""" + line_id + """();
                        }
                    }
                }, 1000);
            }
            monitor_page_""" + line_id + """();

            function load_geomap_""" + line_id + """() {
                Highcharts.stockChart('geo_""" + line_id + """_subchart', {
                    title: {
                        text: '""" + metric + """ for """ + state[:-1] + """'
                    },
                    chart: {
                        zoomType: 'x'
                    },
                    credits: {
                        enabled: false,
                    },
                    xAxis: {
                        plotLines: [{ 
                            color: '#FD35BE',
                            width: 2,
                            value: """ + str((datetime(2016, 11, 3, 0, 0, 0) -  datetime.utcfromtimestamp(0)).total_seconds() * 1000.0) + """,
                            label: {
                                text: 'Official announcement',
                                rotation: 0,
                                align: 'left',
                                x: -22,
                                y: -7,
                                style: {
                                    color: '#FD35BE',
                                    fontSize: 10,

                                }
                            }
                        },
                        { 
                            color: '#EB7A12',
                            width: 2,
                            value: """ + str((datetime(2016, 9, 24, 0, 0, 0) -  datetime.utcfromtimestamp(0)).total_seconds() * 1000.0) + """,
                            label: {
                                text: 'Media reports',
                                rotation: 0,
                                align: 'right',
                                x: -10,
                                y: -7,
                                style: {
                                    color: '#EB7A12',
                                    fontSize: 10,
                                }
                            }
                        }],
                        plotBands: [{
                            color: '#c7daeb',
                            from: """ + str((datetime(2017, 4, 1, 0, 0, 0) -  datetime.utcfromtimestamp(0)).total_seconds() * 1000.0) + """,
                            to: """ + str((datetime(2017, 9, 30, 0, 0, 0) -  datetime.utcfromtimestamp(0)).total_seconds() * 1000.0) + """,
                            label: {
                                text: 'Closure of Hazelwood',
                                rotation: 0,
                                y: -7,
                                style: {
                                    fontSize: 10,
                                }
                            }
                      }],
                    },
                    yAxis: {
                        title: {
                            text: '""" + metric + """'
                        },
                        opposite: false
                    },
                    plotOptions: {
                        series: {
                            dataGrouping: {
                                units: [['month', [1]]],
                            }
                        }
                    },
                    legend: {
                        enabled: false
                    },
                    rangeSelector: {
                        enabled: false
                    },
                    navigator: {
                        enabled: false
                    },
                    scrollbar: {
                        enabled: false
                    },
                    lang: {
                        numericSymbols: [ "k" , "M" , "B" , "T" , "Qd" , "Qn"]
                    },
                    series: [{
                        name: '""" + metric + """',
                        data: """ + str(state_data[metric][state]) + """,
                        type: '""" + str(line_chart_type[metric]) + """'
                    }]
                });
                $('#geo_""" + line_id + """_subchart').hide();
            }
            </script>
            """))

1.1 Impact on wholesale electricity prices

The analysis of prices (RRP, VWAP and Peak) suggest that Hazelwood’s withdrawal has an impact on wholesale electricity prices.

In [13]:
def divide_by_30(data):
    for row in data:
        row[1] = row[1] / 30.0
    return data

chart = Highstock(width=900, height=500)
chart.set_dict_options(default_chart_options)
chart.set_dict_options({
    # custom
    'title': {
        'text': 'Wholesale Electricity Prices (RRP, VWAP and Peak)',
        'style': {
            'fontWeight': 'bold',
            'color': '#616161',
            'fontSize': 20
        }
    },
    'xAxis': {
        'plotLines': [{ 
            'color': '#FD35BE',
            'width': 2,
            'value': (datetime(2016, 11, 3, 0, 0, 0) -  datetime.utcfromtimestamp(0)).total_seconds() * 1000.0,
            'label': {
                'text': 'Official announcement',
                'rotation': 0,
                'align': 'left',
                'x': -22,
                'y': -7,
                'style': {
                    'color': '#FD35BE',
                    'fontSize': 10,
                    
                }
            }
        },
        { 
            'color': '#EB7A12',
            'width': 2,
            'value': (datetime(2016, 9, 24, 0, 0, 0) -  datetime.utcfromtimestamp(0)).total_seconds() * 1000.0,
            'label': {
                'text': 'Media reports',
                'rotation': 0,
                'align': 'right',
                'x': -10,
                'y': -7,
                'style': {
                    'color': '#EB7A12',
                    'fontSize': 10,
                }
            }
        }],
        'plotBands': [{
            'color': '#c7daeb',
            'from': (datetime(2017, 4, 1, 0, 0, 0) -  datetime.utcfromtimestamp(0)).total_seconds() * 1000.0,
            'to': (datetime(2017, 9, 30, 0, 0, 0) -  datetime.utcfromtimestamp(0)).total_seconds() * 1000.0,
            'label': {
                'text': 'Closure of Hazelwood',
                'rotation': 0,
                'y': -7,
                'style': {
                    'fontSize': 10,
                }
            }
      }],
    },
    'yAxis': [
        {
            'title': {
                'text': 'Price ($)',
                'style': {
                    'fontWeight': 'bold',
                    'color': '#000000',
                    'fontSize': 15
                }
            },
            'offset': 30,
            'opposite': False
        },
        {
            'title': {
                'text': 'Peak price ($)',
                'style': {
                    'fontWeight': 'bold',
                    'color': '#000000',
                    'fontSize': 15
                }
            }
        },
    ],
    'legend': {
        'reversed': True
    },
    'plotOptions': {
        'series': {
            'dataGrouping': {
                'units': [['month', [1]]]
            }
        }
    },
})


series1 = read_csv('daily_average_price_2015_01_2017_09.csv', 'settlement_date')
series2 = read_csv('daily_vwap_2015_01_2017_09.csv', 'settlement_date')
series3 = read_csv('daily_average_peak_price_2015_01_2017_09.csv', '') #COLUMN HAS NO NAME?

for state in sorted(series1, key=lambda x: (x != 'VIC1', x)):
    visible = state == 'VIC1'
    chart.add_data_set(divide_by_30(series1[state]), name=state[:-1] + ' - Daily average RRP', type='column', visible=visible)
    chart.add_data_set(divide_by_30(series2[state]), name=state[:-1] + ' - VWAP', type='column', visible=visible)
    chart.add_data_set(series3[state], name=state[:-1] + ' - Average peak price', type='line', lineWidth=3, yAxis=1, visible=visible)
chart
Out[13]: