Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

v1.4.1 #298

Closed
wants to merge 33 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
bbe7590
Coverage of unimplemented method in fix_nonpositive_semidefinite.
SeaPea1 Feb 8, 2021
632afa9
Coverage of DataFrame parameter to fix_nonpositive_semidefinite.
SeaPea1 Feb 8, 2021
1aa0252
Coverage improvements.
SeaPea1 Feb 9, 2021
0eb8fa8
Add a FIXME to make it obvious where not covered (and broken).
SeaPea1 Feb 10, 2021
f2acf3f
Improve test coverage of plotting.
SeaPea1 Feb 10, 2021
fe9a2ca
Improve test coverage of DiscreteAllocation.
SeaPea1 Feb 13, 2021
a536331
Improve test coverage of BlackLitterman.
SeaPea1 Feb 13, 2021
70235ef
Black file.
SeaPea1 Feb 13, 2021
c9ca6ad
Correct type of weight_bounds, as does accept list of tuples. This w…
SeaPea1 Feb 15, 2021
e9e9aae
Misc coverage improvements.
SeaPea1 Feb 15, 2021
75e6730
Merging
SeaPea1 Feb 16, 2021
28ef225
Fix test errors in merged file.
SeaPea1 Feb 16, 2021
7ffd1b0
minimal setup
phschiele Feb 16, 2021
9fe7deb
Move Dockerfile
phschiele Feb 16, 2021
4a10499
Change Dockerfile reference path
phschiele Feb 16, 2021
b20d510
Merge pull request #285 from SeaPea1/master
robertmartin8 Feb 17, 2021
dc04a2e
Merge branch 'master' into docker-versioning
phschiele Feb 17, 2021
6ed1ded
deprecate min cov det #294
robertmartin8 Feb 18, 2021
59ad7fa
Merge pull request #286 from phschiele/docker-versioning
robertmartin8 Feb 18, 2021
cc48e5f
added cov to local workflow
robertmartin8 Feb 18, 2021
1ee2e24
Merge branch 'v1.4.1' of https://github.com/robertmartin8/PyPortfolio…
robertmartin8 Feb 18, 2021
982132b
Merge branch 'master' into v1.4.1
robertmartin8 Feb 18, 2021
50645a8
Merge branch 'master' into v1.4.1
robertmartin8 Feb 18, 2021
627dc62
Made cvxopt optional (#289)
robertmartin8 Feb 18, 2021
eb5f5ee
updated issue templates
robertmartin8 Feb 18, 2021
057114a
improved test cov to 100%
robertmartin8 Feb 18, 2021
388678e
restructured module (fixes #290)
robertmartin8 Feb 19, 2021
72b8bdb
restructured submodule (fixes #290)
robertmartin8 Feb 19, 2021
746abab
reorganised docs
robertmartin8 Feb 19, 2021
b34179b
removed makefile
robertmartin8 Feb 19, 2021
9f4aef1
consistent use of "optimization"
robertmartin8 Feb 19, 2021
8debd04
improved tests
robertmartin8 Feb 19, 2021
1073641
doc fixes
robertmartin8 Feb 19, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 10 additions & 14 deletions .dockerignore
Original file line number Diff line number Diff line change
@@ -1,15 +1,11 @@
#https://stackoverflow.com/questions/28097064/dockerignore-ignore-everything-except-a-file-and-the-dockerfile
# Environments
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/

# Ignore Everything
**

!pypfopt
!tests
!setup.py
!README.md
!requirements.txt
!binder
!cookbook

**/__pycache__
**/*.pyc
*/__pycache__
*/*.pyc
4 changes: 2 additions & 2 deletions .github/ISSUE_TEMPLATE/bug.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,8 @@ A clear and concise description of what the bug is.
**Expected behavior**
A clear and concise description of what you expected to happen.

**Screenshots**
If applicable, add screenshots to help explain your problem.
**Code sample**
Add a minimal reproducible example (see [here](https://stackoverflow.com/help/minimal-reproducible-example)).

**Operating system, python version, PyPortfolioOpt version**
e.g MacOS 10.146, python 3.7.3, PyPortfolioOpt 1.2.6
Expand Down
4 changes: 2 additions & 2 deletions .github/ISSUE_TEMPLATE/feature_request.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,8 @@ assignees: ''
**Is your feature request related to a problem?**
A clear and concise description of what the problem is.

**Describe the solution you'd like**
A clear and concise description of what you want to happen.
**Describe the feature you'd like**
A clear description of the feature you want, or a link to the textbook/article describing the feature.

**Additional context**
Add any other context or screenshots about the feature request here.
2 changes: 2 additions & 0 deletions .github/ISSUE_TEMPLATE/help-needed.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,5 +10,7 @@ assignees: ''
**What are you trying to do?**
Clear description of the problem you are trying to solve with PyPortfolioOpt

**What have you tried?**

**What data are you using?**
What asset class, how many assets, how many data points. Preferably provide a sample of the dataset as a csv attachment.
2 changes: 1 addition & 1 deletion .github/ISSUE_TEMPLATE/installation-error.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,5 +16,5 @@ pip install pyportfolioopt
**Error message**

```
Copy paste the terminal message here
Copy paste the terminal message inside the backticks.
```
12 changes: 12 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,20 @@ DEV/
.pytest_cache/
.vscode/

# Environments
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/

pip-selfcheck.json

.coverage
.coverage*

html-coverage
html-report

Expand Down
4 changes: 2 additions & 2 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ Please refer to the roadmap for a list of areas that I think PyPortfolioOpt coul
from. In addition, the following is always welcome::

- Improve performance of existing code (but not at the cost of readability) – are there any nice numpy tricks I've missed?
- Add new optimisation objective functions. For example, if you think that the best performance metric has not been included, write it into a function (or suggest it in [Issues](https://github.com/robertmartin8/PyPortfolioOpt/issues) and I will have a go).
- Add new optimization objective functions. For example, if you think that the best performance metric has not been included, write it into a function (or suggest it in [Issues](https://github.com/robertmartin8/PyPortfolioOpt/issues) and I will have a go).
- Help me write more tests! If you are someone learning about quant finance and/or unit testing in python, what better way to practice than to write some tests on an open-source project! Feel free to check for edge cases, or test performance on a dataset with more stocks.

## Guidelines
Expand Down Expand Up @@ -37,7 +37,7 @@ If you have questions unrelated to the project, drop me an email – contact det

## Bugs/issues

If you find any bugs or the portfolio optimisation is not working as expected, feel free to [raise an issue](https://github.com/robertmartin8/PyPortfolioOpt/issues). I would ask that you provide the following information in the issue:
If you find any bugs or the portfolio optimization is not working as expected, feel free to [raise an issue](https://github.com/robertmartin8/PyPortfolioOpt/issues). I would ask that you provide the following information in the issue:

- Descriptive title so that other users can see the existing issues
- Operating system, python version, and python distribution (optional).
Expand Down
26 changes: 0 additions & 26 deletions Dockerfile

This file was deleted.

54 changes: 0 additions & 54 deletions Makefile

This file was deleted.

38 changes: 20 additions & 18 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,8 +26,8 @@

<!-- content -->

PyPortfolioOpt is a library that implements portfolio optimisation methods, including
classical mean-variance optimisation techniques and Black-Litterman allocation, as well as more
PyPortfolioOpt is a library that implements portfolio optimization methods, including
classical mean-variance optimization techniques and Black-Litterman allocation, as well as more
recent developments in the field like shrinkage and Hierarchical Risk Parity, along with
some novel experimental features like exponentially-weighted covariance matrices.

Expand All @@ -49,14 +49,14 @@ Head over to the [documentation on ReadTheDocs](https://pyportfolioopt.readthedo
- [For development](#for-development)
- [A quick example](#a-quick-example)
- [What's new](#whats-new)
- [An overview of classical portfolio optimisation methods](#an-overview-of-classical-portfolio-optimisation-methods)
- [An overview of classical portfolio optimization methods](#an-overview-of-classical-portfolio-optimization-methods)
- [Features](#features)
- [Expected returns](#expected-returns)
- [Risk models (covariance)](#risk-models-covariance)
- [Objective functions](#objective-functions)
- [Adding constraints or different objectives](#adding-constraints-or-different-objectives)
- [Black-Litterman allocation](#black-litterman-allocation)
- [Other optimisers](#other-optimisers)
- [Other optimizers](#other-optimizers)
- [Advantages over existing implementations](#advantages-over-existing-implementations)
- [Project principles and design decisions](#project-principles-and-design-decisions)
- [Testing](#testing)
Expand Down Expand Up @@ -124,7 +124,7 @@ df = pd.read_csv("tests/resources/stock_prices.csv", parse_dates=True, index_col
mu = expected_returns.mean_historical_return(df)
S = risk_models.sample_cov(df)

# Optimise for maximal Sharpe ratio
# Optimize for maximal Sharpe ratio
ef = EfficientFrontier(mu, S)
raw_weights = ef.max_sharpe()
cleaned_weights = ef.clean_weights()
Expand Down Expand Up @@ -206,23 +206,23 @@ As of v1.2.0:
new plots. All other plotting functions (scattered in different classes) have been retained,
but are now deprecated.

## An overview of classical portfolio optimisation methods
## An overview of classical portfolio optimization methods

Harry Markowitz's 1952 paper is the undeniable classic, which turned portfolio optimisation from an art into a science. The key insight is that by combining assets with different expected returns and volatilities, one can decide on a mathematically optimal allocation which minimises the risk for a target return – the set of all such optimal portfolios is referred to as the **efficient frontier**.
Harry Markowitz's 1952 paper is the undeniable classic, which turned portfolio optimization from an art into a science. The key insight is that by combining assets with different expected returns and volatilities, one can decide on a mathematically optimal allocation which minimises the risk for a target return – the set of all such optimal portfolios is referred to as the **efficient frontier**.

<center>
<img src="https://github.com/robertmartin8/PyPortfolioOpt/blob/master/media/efficient_frontier_white.png" style="width:60%;"/>
</center>

Although much development has been made in the subject, more than half a century later, Markowitz's core ideas are still fundamentally important and see daily use in many portfolio management firms.
The main drawback of mean-variance optimisation is that the theoretical treatment requires knowledge of the expected returns and the future risk-characteristics (covariance) of the assets. Obviously, if we knew the expected returns of a stock life would be much easier, but the whole game is that stock returns are notoriously hard to forecast. As a substitute, we can derive estimates of the expected return and covariance based on historical data – though we do lose the theoretical guarantees provided by Markowitz, the closer our estimates are to the real values, the better our portfolio will be.
The main drawback of mean-variance optimization is that the theoretical treatment requires knowledge of the expected returns and the future risk-characteristics (covariance) of the assets. Obviously, if we knew the expected returns of a stock life would be much easier, but the whole game is that stock returns are notoriously hard to forecast. As a substitute, we can derive estimates of the expected return and covariance based on historical data – though we do lose the theoretical guarantees provided by Markowitz, the closer our estimates are to the real values, the better our portfolio will be.

Thus this project provides four major sets of functionality (though of course they are intimately related)

- Estimates of expected returns
- Estimates of risk (i.e covariance of asset returns)
- Objective functions to be optimised
- Optimisers.
- Objective functions to be optimized
- Optimizers.

A key design goal of PyPortfolioOpt is **modularity** – the user should be able to swap in their
components while still making use of the framework that PyPortfolioOpt provides.
Expand Down Expand Up @@ -253,7 +253,7 @@ The covariance matrix encodes not just the volatility of an asset, but also how
- an unbiased estimate of the covariance matrix
- relatively easy to compute
- the de facto standard for many years
- however, it has a high estimation error, which is particularly dangerous in mean-variance optimisation because the optimiser is likely to give excess weight to these erroneous estimates.
- however, it has a high estimation error, which is particularly dangerous in mean-variance optimization because the optimizer is likely to give excess weight to these erroneous estimates.
- Semicovariance: a measure of risk that focuses on downside variation.
- Exponential covariance: an improvement over sample covariance that gives more weight to recent data
- Covariance shrinkage: techniques that involve combining the sample covariance matrix with a structured estimator, to reduce the effect of erroneous weights. PyPortfolioOpt provides wrappers around the efficient vectorised implementations provided by `sklearn.covariance`.
Expand All @@ -280,7 +280,7 @@ The covariance matrix encodes not just the volatility of an asset, but also how

### Adding constraints or different objectives

- Long/short: by default all of the mean-variance optimisation methods in PyPortfolioOpt are long-only, but they can be initialised to allow for short positions by changing the weight bounds:
- Long/short: by default all of the mean-variance optimization methods in PyPortfolioOpt are long-only, but they can be initialised to allow for short positions by changing the weight bounds:

```python
ef = EfficientFrontier(mu, S, weight_bounds=(-1, 1))
Expand All @@ -299,7 +299,7 @@ ef.efficient_return(target_return=0.2, market_neutral=True)
ef = EfficientFrontier(mu, S, weight_bounds=(0, 0.1))
```

One issue with mean-variance optimisation is that it leads to many zero-weights. While these are
One issue with mean-variance optimization is that it leads to many zero-weights. While these are
"optimal" in-sample, there is a large body of research showing that this characteristic leads
mean-variance portfolios to underperform out-of-sample. To that end, I have introduced an
objective function that can reduce the number of negligible weights for any of the objective functions. Essentially, it adds a penalty (parameterised by `gamma`) on small weights, with a term that looks just like L2 regularisation in machine learning. It may be necessary to try several `gamma` values to achieve the desired number of non-negligible weights. For the test portfolio of 20 securities, `gamma ~ 1` is sufficient
Expand Down Expand Up @@ -328,14 +328,16 @@ ef = EfficientFrontier(rets, S)
ef.max_sharpe()
```

### Other optimisers
### Other optimizers

The features above mostly pertain to solving efficient frontier optimisation problems via quadratic programming (though this is taken care of by `cvxpy`). However, we offer different optimisers as well:
The features above mostly pertain to solving mean-variance optimization problems via quadratic programming (though this is taken care of by `cvxpy`). However, we offer different optimizers as well:

- Mean-semivariance optimization
- Mean-CVaR optimization
- Hierarchical Risk Parity, using clustering algorithms to choose uncorrelated assets
- Markowitz's critical line algorithm (CLA)

Please refer to the [documentation](https://pyportfolioopt.readthedocs.io/en/latest/OtherOptimisers.html) for more.
Please refer to the [documentation](https://pyportfolioopt.readthedocs.io/en/latest/OtherOptimizers.html) for more.

## Advantages over existing implementations

Expand All @@ -350,10 +352,10 @@ Please refer to the [documentation](https://pyportfolioopt.readthedocs.io/en/lat

## Project principles and design decisions

- It should be easy to swap out individual components of the optimisation process
- It should be easy to swap out individual components of the optimization process
with the user's proprietary improvements.
- Usability is everything: it is better to be self-explanatory than consistent.
- There is no point in portfolio optimisation unless it can be practically
- There is no point in portfolio optimization unless it can be practically
applied to real asset prices.
- Everything that has been implemented should be tested.
- Inline documentation is good: dedicated (separate) documentation is better.
Expand Down
20 changes: 0 additions & 20 deletions binder/Dockerfile

This file was deleted.

7 changes: 0 additions & 7 deletions binder/jupyter_notebook_config.py

This file was deleted.

2 changes: 1 addition & 1 deletion cookbook/1-RiskReturnModels.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -240,7 +240,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"The exponential moving average is marginally better than the others, but the improvement is almost unnoticeable. We also note that the mean absolute deviations are above 25%, meaning that if your expected annual returns are 10%, on average the realised annual return could be anywhere from a 15% loss to a 35% gain. This is a massive range, and gives some context to the advice in the docs suggesting that you optimise without providing an estimate of returns."
"The exponential moving average is marginally better than the others, but the improvement is almost unnoticeable. We also note that the mean absolute deviations are above 25%, meaning that if your expected annual returns are 10%, on average the realised annual return could be anywhere from a 15% loss to a 35% gain. This is a massive range, and gives some context to the advice in the docs suggesting that you optimize without providing an estimate of returns."
]
},
{
Expand Down
Loading