Python for Finance: Calculate and Plot S&P 500 Daily Returns

In this post, we will write a Python script that will calculate S&P 500 historical returns. The majority of investors are always trying to find an answer to a simple question, how the market will do in the future.

Obviously, no one knows the answer and therefore investors and financial analysts spend hours and hours trying to come up with a best estimate for future stock prices. A simple approach is to look at historical market returns and think if the market will follow historical price returns.

Python for finance can be a great ally to support us with this analysis. We will show how by following below steps using S&P 500 historical prices:

  • Retrieve market data with Python and Pandas
  • Calculate market daily returns
  • Plot the market daily return in a chart

After reading this article, you will be able to analyse historical market returns based on S&P 500 historical prices. Let’s start.

Photo by Stephen Dawson on Unsplash
Photo by Stephen Dawson on Unsplash

Retrieving S&P 500 Historical Returns with Python

First of all, for any analysis we need to have data. Luckily for us, with Python we can download hundred of data points within seconds. For our anylsis, we need to retrieve market data in form of stock prices.

As a proxy of market data, we will use the S&P 500 index which tracks the performance of the 500 largest companies in the United States.

The S&P 500 daily data can be extracted using Pandas and Pandas DataReader. Pandas DataReader is a great package to access data remotely with Pandas. To use this package, you need to have a version of pandas higher than 0.19.2.

Within Pandas DataReader we have multiple sources where we can download data to perform multiple financial analysis with Python. We can get for example stock data or economic indicators from reliable providers. Check the Pandas DataReader documentation for more information.

For our analysis, we will retrieve data from the FRED datasource. First, we need to import Pandas, datetime and Pandas_datareader packages.

Next, we can simply define a date range and use the web.DataReader method to download the data in the form of a Pandas DataFrame and store it in a variable name SP500.

We passed as arguments of the DataReader method the name of the dataset that we want to download (i.e. s&p 500 historical prices), the provider or source of the data (‘Fred’) and the start and ending dates:

import pandas as pd
#if you get an error after executing the code, try adding below:
pd.core.common.is_list_like = pd.api.types.is_list_like

import pandas_datareader.data as web
import datetime

start = datetime.datetime(2010, 1, 1)
end = datetime.datetime(2020, 1, 27)

SP500 = web.DataReader(['sp500'], 'fred', start, end)

After running the code, we have below Pandas DataFrame showing the historical S&P 500 daily prices:

Python for Finance: S&P 500 prices

Calculating S&P 500 Daily Returns

Great, we have the S&P 500 historical prices from the last 10 years in a Pandas DataFrame. Now, we are ready to calculate the S&P 500 daily returns from this period of time and add them to our DataFrame as a new column that we will call daily_return.

To calculate the rate of return, we can use below formula where P1 refers to current price. P0 refers to the initial price or price that we take as reference to calculate the rate of return.

Python Calculate Rate of Return
Simple Rate of Return
SP500['daily_return'] = (SP500['sp500']/ SP500['sp500'].shift(1)) -1

#Drop all Not a number values using drop method.
SP500.dropna(inplace = True)

What shift method does is to shift our index by the number provided as an argument. In our case, we are shifting each of the values in the sp500 column by one. Meaning that we are diving the current day S&P 500 price by the previous day S&P 500 price.

Lets print the last five values of our SP500 DataFrame to see what we get:

s&p 500 historical returns
S&P 500 Daily Return

Plotting the S&P 500 Historical Returns

We have our S&P 500 prices and daily returns ready to plot with Python. Lets plot the daily returns first.

Plotting with Python and Matplotlib is super easy, we only need to select the daily_return column from our SP500 DataFrame and use the method plot.

SP500['daily_return'].plot(title='S&P 500 daily returns')
Python for Finance: S&P 500 Returns
Python for Finance: S&P 500 Daily Returns

Nice! We can easily identify in the graph some very useful information. For example, we can see that the worst daily return for the S&P 500 index was in 2011 with a daily return of -7%.

Similarly, we can see that the best day for the S&P 500 index was in the middle of 2019 with around 5% daily return.

In addition to the daily returns, we can also plot the S&P 500 price evolution for the last 10 years. Here we can observe a clear upward trend.

SP500['sp500'].plot(title='S&P 500 Price')
S&P 500 Historical Prices
S&P 500 Historical Prices

Wrapping Up

In this post on Python for Finance, we have retrieved S&P 500 historical prices in order to calculate and plot S&P 500 daily returns for the index.

Plotting daily market returns is a great way to visualise stock returns for any given period of time. You can try it out with any other stock or index.

I really hope you have enjoyed the story! Check out my other articles on Python for Finance in the Data Science section of the blog.

If you have questions or anything is not clear, feel free to leave a comment. I also have a Youtube video explaining step by step the code used in this tutorial:

Plotting SP500 daily returns with Python

Thank you for reading!