andrewkdinh
/
fund-indicators
mirror of https://github.com/andrewkdinh/fund-indicators.git
1
0
Fork 0
Code Issues Releases Wiki Activity

Added code of conduct and contributing guidelines. 

+ Bug fixes
This commit is contained in:
Andrew Dinh 2019-04-08 07:19:39 -07:00
parent 4fd66eab52
commit 18730bcd43
6 changed files with 367 additions and 124 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

77
CODE-OF-CONDUCT.md Normal file
View File

@ -0,0 +1,77 @@
# Contributor Covenant Code of Conduct
## Our Pledge
In the interest of fostering an open and welcoming environment, we as
contributors and maintainers pledge to making participation in our project and
our community a harassment-free experience for everyone, regardless of age, body
size, disability, ethnicity, sex characteristics, gender identity and expression,
level of experience, education, socio-economic status, nationality, personal
appearance, race, religion, or sexual identity and orientation.
## Our Standards
Examples of behavior that contributes to creating a positive environment
include:
* Using welcoming and inclusive language
* Being respectful of differing viewpoints and experiences
* Gracefully accepting constructive criticism
* Focusing on what is best for the community
* Showing empathy towards other community members
Examples of unacceptable behavior by participants include:
* The use of sexualized language or imagery and unwelcome sexual attention or
advances
* Trolling, insulting/derogatory comments, and personal or political attacks
* Public or private harassment
* Publishing others' private information, such as a physical or electronic
address, without explicit permission
* Other conduct which could reasonably be considered inappropriate in a
professional setting
## Our Responsibilities
Project maintainers are responsible for clarifying the standards of acceptable
behavior and are expected to take appropriate and fair corrective action in
response to any instances of unacceptable behavior.
Project maintainers have the right and responsibility to remove, edit, or
reject comments, commits, code, wiki edits, issues, and other contributions
that are not aligned to this Code of Conduct, or to ban temporarily or
permanently any contributor for other behaviors that they deem inappropriate,
threatening, offensive, or harmful.
## Scope
This Code of Conduct applies within all project spaces, and it also applies when
an individual is representing the project or its community in public spaces.
Examples of representing a project or community include using an official
project e-mail address, posting via an official social media account, or acting
as an appointed representative at an online or offline event. Representation of
a project may be further defined and clarified by project maintainers.
## Enforcement
Instances of abusive, harassing, or otherwise unacceptable behavior may be
reported by contacting the project team at fund-indicators@andrewkdinh.com. All
complaints will be reviewed and investigated and will result in a response that
is deemed necessary and appropriate to the circumstances. The project team is
obligated to maintain confidentiality with regard to the reporter of an incident.
Further details of specific enforcement policies may be posted separately.
Project maintainers who do not follow or enforce the Code of Conduct in good
faith may face temporary or permanent repercussions as determined by other
members of the project's leadership.
## Attribution
This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4,
available at https://www.contributor-covenant.org/version/1/4/code-of-conduct.html
[homepage]: https://www.contributor-covenant.org
For answers to common questions about this code of conduct, see
https://www.contributor-covenant.org/faq

37
CONTRIBUTING.md Normal file
View File

@ -0,0 +1,37 @@
# How to Contribute
**Did you come here to read what you should do before creating an issue?** Scroll down!
## How you can help
There are many ways to contribute to fund-indicators:
- Help with testing and reporting bugs
- Help write documentation
- Help implement new features
- New sources of data
- Better web scraping
- Better statistical measures
- Multithreading (asynchronous requests)
## Filing an issue
Thanks for wanting to help out with squashing bugs and more by filing an issue.
There are a few things you might consider when filing your issue:
- What made the issue/bug appear? (steps to reproduce)
- Do you have the latest version of this program?
## Getting started with development
To develop on fund-indicators you'll probably want to:
1. Read the [code of conduct](https://github.com/andrewkdinh/fund-indicators/CODE_OF_CONDUCT.md)
2. Install normally, as documented in the [README](https://github.com/andrewkdinh/fund-indicators#quickstart)
3. Read the [documentation](https://github.com/andrewkdinh/fund-indicators/wiki) (although it's not much)
4. Read the section below to understand the ways you could best help this project
## Questions?
If you have any questions, please email fund-indicators@andrewkdinh.com.

46
Functions.py
View File

@ -11,7 +11,7 @@ def getNearest(items, pivot):
def stringToDate(date):
from datetime import datetime
#datetime_object = datetime.strptime('Jun 1 2005 1:33PM', '%b %d %Y %I:%M%p')
# datetime_object = datetime.strptime('Jun 1 2005 1:33PM', '%b %d %Y %I:%M%p')
datetime_object = datetime.strptime(date, '%Y-%m-%d').date()
return(datetime_object)
@ -57,17 +57,17 @@ def strintIsFloat(s):
def fromCache(r):
import requests_cache
from termcolor import colored, cprint
if r.from_cache == True:
if r.from_cache is True:
cprint('(Response taken from cache)', 'white', attrs=['dark'])
return
def getJoke():
import requests
import sys
from termcolor import colored, cprint
import requests_cache
from halo import Halo
import sys
with requests_cache.disabled():
'''
f = requests.get('https://official-joke-api.appspot.com/jokes/random').json()
@ -81,7 +81,7 @@ def getJoke():
cprint('Get: ' + url, 'white', attrs=['dark'])
with Halo(spinner='dots'):
f = requests.get('https://icanhazdadjoke.com/',
f = requests.get(url,
headers=headers).json()
print('')
print(colored(f['joke'], 'green'))
@ -118,7 +118,7 @@ def checkPackages(listOfPackages):
def checkPythonVersion():
import platform
#print('Checking Python version')
# print('Checking Python version')
i = platform.python_version()
r = i.split('.')
k = float(''.join((r[0], '.', r[1])))
@ -135,7 +135,7 @@ def isConnected():
import socket # To check internet connection
try:
# connect to the host -- tells us if the host is actually reachable
socket.create_connection(("www.andrewkdinh.com", 80))
socket.create_connection(('1.1.1.1', 53))
print('Internet connection is good')
return True
except OSError:
@ -199,9 +199,43 @@ def keyInDict(dict, key):
return False
def getWeather():
import requests
from termcolor import colored, cprint
import requests_cache
from halo import Halo
import sys
sys.path.insert(0, './modules')
with requests_cache.disabled():
url = 'https://wttr.in?format=3'
cprint('Get: ' + url, 'white', attrs=['dark'])
with Halo(spinner='dots'):
f = requests.get(url)
print('')
print(colored('Current weather in ' + f.text, 'green'), end='')
def main():
exit()
if __name__ == "__main__":
main()
'''
Copyright (C) 2019 Andrew Dinh
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>.
'''

28
README.md
View File

@ -1,45 +1,55 @@
# fund-indicators
[![License](https://img.shields.io/github/license/andrewkdinh/fund-indicators.svg)](https://raw.githubusercontent.com/andrewkdinh/fund-indicators/master/LICENSE)
[![](https://img.shields.io/github/last-commit/andrewkdinh/fund-indicators.svg)](https://github.com/andrewkdinh/fund-indicators/commits/master)
[![Latest Commits](https://img.shields.io/github/last-commit/andrewkdinh/fund-indicators.svg)](https://github.com/andrewkdinh/fund-indicators/commits/master)
![](https://img.shields.io/github/languages/top/andrewkdinh/fund-indicators.svg)
![](https://img.shields.io/github/languages/code-size/andrewkdinh/fund-indicators.svg)
[![CII Best Practices](https://bestpractices.coreinfrastructure.org/projects/2667/badge)](https://bestpractices.coreinfrastructure.org/projects/2667)
A project to determine relationships between mutual funds and different factors.
A project to determine relationships between mutual fund performance and different factors.
[![asciicast demo](https://asciinema.org/a/jLmZapnMFGCRiiSUITY21erLW.svg)](https://asciinema.org/a/jLmZapnMFGCRiiSUITY21erLW?autoplay=1&preload=1)
Calculates relationships between: Previous performance, Alpha, Sharpe Ratio, Sortino Ratio
and Expense ratios, Turnover, Market Capitalization (Asset Size), Persistence
Give it a try at [repl.run](https://fund-indicators.andrewkdinh.repl.run) or [repl.it](https://repl.it/@andrewkdinh/fund-indicators)
Give it a try at [repl.run](https://fund-indicators.andrewkdinh.repl.run) or [repl.it](https://repl.it/@andrewkdinh/fund-indicators).
## Key Features
- 100% automated
- Uses multiple API's in case another fails
- Uses multiple API's in the case another fails
- Caches http requests for future runs
- Scrapes data from Yahoo Finance
- Color-coded for easy viewing
- Optional graphs to easily visualize linear regression results
- A new joke every time it runs
- A new joke every time
- Cross-platform (tested on Windows and Linux)
## Quickstart
```shell
git clone https://github.com/andrewkdinh/fund-indicators.git && cd fund-indicators
pip install -r requirements.txt
python main.py
```
Pre-chosen stocks listed in `stocks.txt`
- Common mutual funds are listed in `stocks.txt`
- Configure and rename `config.example.json` to `config.json` if you would like to skip beginning questions (only for advanced users)
### Contributing
Want to help? Great! Check out the [CONTRIBUTING.md](https://github.com/andrewkdinh/fund-indicators/CONTRIBUTING.md) file!
## Credits
This project uses a wide variety of open-source projects
This project utilizes a wide variety of open-source projects:
- [NumPy](https://github.com/numpy/numpy), [Termcolor](https://github.com/hfeeki/termcolor), [Beautiful Soup](https://launchpad.net/beautifulsoup), [yahoofinancials](https://github.com/JECSand/yahoofinancials), [requests-cache](https://github.com/reclosedev/requests-cache), [halo](https://github.com/manrajgrover/halo)
- [NumPy](https://github.com/numpy/numpy), [Termcolor](https://github.com/hfeeki/termcolor), [Beautiful Soup](https://launchpad.net/beautifulsoup), [yahoofinancials](https://github.com/JECSand/yahoofinancials), [requests-cache](https://github.com/reclosedev/requests-cache), [halo](https://github.com/manrajgrover/halo), [matplotlib](https://github.com/matplotlib/matplotlib), [asciinema](https://github.com/asciinema/asciinema)
And thank you to those that have helped me with the idea and product:
- Amber Bruce, [Alex Stoykov](http://stoykov.us/), Doug Achterman, [Stack Overflow](https://stackoverflow.com)
Created by Andrew Dinh from Dr. TJ Owens Gilroy Early College Academy
Licensed under [GPL-3.0](https://raw.githubusercontent.com/andrewkdinh/fund-indicators/master/LICENSE) | Copyright (C) 2019 Andrew Dinh

10
config.example.json
View File

@ -6,7 +6,6 @@
"Check Internet Connection": false,
"Get Joke": true,
"Benchmark": "SPY",
"Method": "Kiplinger",
"Time Frame": 60,
"Indicator": "Expense Ratio",
"Remove Outliers": true,
@ -40,13 +39,6 @@
"VTHR",
"EFG"
],
"Method": [
"Read",
"Manual",
"U.S. News",
"Kiplinger",
"TheStreet"
],
"Time Frame": "Any integer",
"Indicator": [
"Expense Ratio",
@ -60,4 +52,4 @@
],
"Sources": "Choose an order out of ['Alpha Vantage', 'Yahoo', 'IEX', 'Tiingo']"
}
}
}

293
main.py
View File

@ -41,7 +41,8 @@ apiTiingo = '2e72b53f2ab4f5f4724c5c1e4d5d4ac0af3f7ca8'
apiTradier = 'n26IFFpkOFRVsB5SNTVNXicE5MPD'
apiQuandl = 'KUh3U3hxke9tCimjhWEF'
# apiIntrinio = 'OmNmN2E5YWI1YzYxN2Q4NzEzZDhhOTgwN2E2NWRhOWNl'
# If you're going to take these API keys and abuse it, you should really reconsider your life priorities
# If you're going to take these API keys and abuse it, you should really
# reconsider your life priorities
'''
API Keys:
@ -77,7 +78,7 @@ class Stock:
# CONFIG
removeOutliers = True
sourceList = ['Alpha Vantage', 'Yahoo', 'IEX', 'Tiingo']
sourceList = ['Yahoo', 'Alpha Vantage', 'IEX', 'Tiingo']
config = 'N/A'
# BENCHMARK VALUES
@ -147,7 +148,7 @@ class Stock:
print("\nFinding all dates given")
allDates = []
for i in range(0, len(loaded_json), 1): # If you want to do oldest first
for i in range(0, len(loaded_json), 1): # For oldest first
# for i in range(len(loaded_json)-1, -1, -1):
line = loaded_json[i]
date = line['date']
@ -157,7 +158,7 @@ class Stock:
# print("\nFinding close values for each date")
values = []
for i in range(0, len(loaded_json), 1): # If you want to do oldest first
for i in range(0, len(loaded_json), 1): # For oldest first
# for i in range(len(loaded_json)-1, -1, -1):
line = loaded_json[i]
value = line['close']
@ -215,7 +216,7 @@ class Stock:
f = requests.get(url, headers=headers)
Functions.fromCache(f)
loaded_json = f.json()
if len(loaded_json) == 1 or f.status_code != 200 or loaded_json['startDate'] == None:
if len(loaded_json) == 1 or f.status_code != 200 or loaded_json['startDate'] is None:
print("Tiingo not available")
return 'N/A'
@ -289,8 +290,8 @@ class Stock:
# Sometimes close value is a None value
i = 0
while i < len(listYahoo[1]):
if Functions.listIndexExists(listYahoo[1][i]) == True:
if listYahoo[1][i] == None:
if Functions.listIndexExists(listYahoo[1][i]) is True:
if listYahoo[1][i] is None:
del listYahoo[1][i]
del listYahoo[0][i]
i = i - 1
@ -381,7 +382,7 @@ class Stock:
print(len(dates), 'dates and', len(closeValues), 'close values')
return datesAndCloseList2
def calcAverageMonthlyReturn(self): # pylint: disable=E0202
def calcAverageMonthlyReturn(self):
# averageMonthlyReturn = (float(self.closeValues[len(self.closeValues)-1]/self.closeValues[0])**(1/(self.timeFrame)))-1
# averageMonthlyReturn = averageMonthlyReturn * 100
averageMonthlyReturn = sum(self.monthlyReturn)/self.timeFrame
@ -687,11 +688,11 @@ class Stock:
break
if marketCap == 0:
somethingWrong = True
if somethingWrong == True:
if somethingWrong is True:
ticker = self.name
yahoo_financials = YahooFinancials(ticker)
marketCap = yahoo_financials.get_market_cap()
if marketCap != None:
if marketCap is not None:
print('(Taken from yahoofinancials)')
print(marketCap)
return int(marketCap)
@ -748,9 +749,6 @@ class Stock:
if s[-1] == '%':
turnover = float(s.replace('%', ''))
break
elif s == 'N/A':
print(self.name, 'has a value of N/A for turnover')
return 'N/A'
if turnover == 0:
print('Something went wrong with scraping turnover')
@ -761,7 +759,7 @@ class Stock:
def indicatorManual(self):
indicatorValueFound = False
while indicatorValueFound == False:
while indicatorValueFound is False:
if Stock.indicator == 'Expense Ratio':
indicatorValue = str(
input(Stock.indicator + ' for ' + self.name + ' (%): '))
@ -780,7 +778,7 @@ class Stock:
'Something is wrong. Indicator was not found. Ending program.', 'white', 'on_red')
exit()
if Functions.strintIsFloat(indicatorValue) == True:
if Functions.strintIsFloat(indicatorValue) is True:
indicatorValueFound = True
return float(indicatorValue)
else:
@ -820,7 +818,7 @@ def benchmarkInit():
benchmark = str(input('Please choose a benchmark from the list: '))
# benchmark = 'SPY' # TESTING
if Functions.stringIsInt(benchmark) == True:
if Functions.stringIsInt(benchmark) is True:
if int(benchmark) <= len(benchmarks) and int(benchmark) > 0:
benchmarkInt = int(benchmark)
benchmark = benchmarks[benchmarkInt-1]
@ -853,60 +851,60 @@ def stocksInit():
print('For simplicity, all of them will be referred to as "stock"')
found = False
while found == False:
while found is False:
print('\nMethods:')
method = 0
methods = ['Read from a file', 'Enter manually',
'U.S. News popular funds (~35)', 'Kiplinger top-performing funds (50)', 'TheStreet top-rated mutual funds (20)']
'Kiplinger top-performing funds (50)',
'TheStreet top-rated mutual funds (20)',
'Money best mutual funds (50)',
'Investors Business Daily best mutual funds (~45)',
'Yahoo top mutual funds (25)']
if Stock.config != 'N/A':
methodsConfig = ['Read', 'Manual',
'U.S. News', 'Kiplinger', 'TheStreet']
for i in range(0, len(methodsConfig), 1):
if Stock.config['Method'] == methodsConfig[i]:
method = i + 1
else:
for i in range(0, len(methods), 1):
print(str(i+1) + '. ' + methods[i])
while method == 0 or method > len(methods):
method = str(input('Which method? '))
if Functions.stringIsInt(method) == True:
method = int(method)
if method == 0 or method > len(methods):
print('Please choose a valid method')
else:
method = 0
print('Please choose a number')
for i in range(0, len(methods), 1):
print(str(i+1) + '. ' + methods[i])
while method == 0 or method > len(methods):
method = str(input('Which method? '))
if Functions.stringIsInt(method) is True:
method = int(method)
if method == 0 or method > len(methods):
print('Please choose a valid method')
else:
method = 0
print('Please choose a number')
print('')
if method == 1:
defaultFiles = ['.gitignore', 'LICENSE', 'main.py', 'Functions.py',
'README.md', 'requirements.txt', 'cache.sqlite', 'yahoofinancials.py', 'termcolor.py', 'README.html', 'config.json', '_test_runner.py'] # Added by repl.it for whatever reason
'README.md', 'requirements.txt', 'cache.sqlite',
'yahoofinancials.py', 'termcolor.py',
'README.html', 'config.json',
'config.example.json', '_test_runner.py']
# Added by repl.it for whatever reason
stocksFound = False
print('\nFiles in current directory (not including default files): ')
print('Files in current directory (without default files): ')
listOfFilesTemp = [f for f in os.listdir() if os.path.isfile(f)]
listOfFiles = []
for files in listOfFilesTemp:
if files[0] != '.' and any(x in files for x in defaultFiles) != True:
if files[0] != '.' and any(x in files for x in defaultFiles) is not True:
listOfFiles.append(files)
for i in range(0, len(listOfFiles), 1):
if listOfFiles[i][0] != '.':
print(str(i+1) + '. ' + listOfFiles[i])
while stocksFound == False:
while stocksFound is False:
fileName = str(input('What is the file number/name? '))
if Functions.stringIsInt(fileName) == True:
if Functions.stringIsInt(fileName) is True:
if int(fileName) < len(listOfFiles)+1 and int(fileName) > 0:
fileName = listOfFiles[int(fileName)-1]
print(fileName)
if Functions.fileExists(fileName) == True:
if Functions.fileExists(fileName) is True:
listOfStocks = []
file = open(fileName, 'r')
n = file.read()
file.close()
s = re.findall(r'[^,;\s]+', n)
for i in s:
if str(i) != '' and Functions.hasNumbers(str(i)) == False:
if str(i) != '' and Functions.hasNumbers(str(i)) is False:
listOfStocks.append(str(i).upper())
stocksFound = True
else:
@ -922,10 +920,10 @@ def stocksInit():
elif method == 2:
isInteger = False
while isInteger == False:
while isInteger is False:
temp = input('\nNumber of stocks to analyze (2 minimum): ')
isInteger = Functions.stringIsInt(temp)
if isInteger == True:
if isInteger is True:
if int(temp) >= 2:
numberOfStocks = int(temp)
else:
@ -939,7 +937,7 @@ def stocksInit():
print('Stock', i + 1, end=' ')
stockName = str(input('ticker: '))
if stockName != '' and Functions.hasNumbers(stockName) == False:
if stockName != '' and Functions.hasNumbers(stockName) is False:
stockName = stockName.upper()
listOfStocks.append(stockName)
listOfStocks[i] = Stock()
@ -949,34 +947,6 @@ def stocksInit():
print('Invalid ticker')
elif method == 3:
listOfStocks = []
url = 'https://money.usnews.com/funds/mutual-funds/most-popular'
headers = {
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36'}
cprint('Get: ' + url, 'white', attrs=['dark'])
with Halo(spinner='dots'):
f = requests.get(url, headers=headers)
Functions.fromCache(f)
raw_html = f.text
soup = BeautifulSoup(raw_html, 'html.parser')
file = open('usnews-stocks.txt', 'w')
r = soup.find_all(
'span', attrs={'class': 'text-smaller text-muted'})
for k in r:
print(k.text.strip(), end=' ')
listOfStocks.append(k.text.strip())
file.write(str(k.text.strip()) + '\n')
file.close()
for i in range(0, len(listOfStocks), 1):
stockName = listOfStocks[i].upper()
listOfStocks[i] = Stock()
listOfStocks[i].setName(stockName)
print('\n' + str(len(listOfStocks)) + ' mutual funds total')
elif method == 4:
listOfStocks = []
url = 'https://www.kiplinger.com/tool/investing/T041-S001-top-performing-mutual-funds/index.php'
headers = {
@ -1003,7 +973,7 @@ def stocksInit():
print('\n' + str(len(listOfStocks)) + ' mutual funds total')
elif method == 5:
elif method == 4:
listOfStocks = []
url = 'https://www.thestreet.com/topic/21421/top-rated-mutual-funds.html'
headers = {
@ -1033,6 +1003,101 @@ def stocksInit():
print('\n' + str(len(listOfStocks)) + ' mutual funds total')
elif method == 5:
listOfStocks = []
url = 'http://money.com/money/4616747/best-mutual-funds-etfs-money-50/'
headers = {
'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.109 Safari/537.36'}
cprint('Get: ' + url, 'white', attrs=['dark'])
with Halo(spinner='dots'):
f = requests.get(url, headers=headers)
Functions.fromCache(f)
raw_html = f.text
soup = BeautifulSoup(raw_html, 'html.parser')
file = open('money.com-stocks.txt', 'w')
r = soup.find_all('td')
for k in r:
t = k.text.strip()
if '(' in t and ')' in t:
t = t.split('(')[1]
t = t.split(')')[0]
print(t, end=' ')
listOfStocks.append(t)
file.write(str(t + '\n'))
file.close()
for i in range(0, len(listOfStocks), 1):
stockName = listOfStocks[i].upper()
listOfStocks[i] = Stock()
listOfStocks[i].setName(stockName)
print('\n' + str(len(listOfStocks)) + ' mutual funds total')
elif method == 6:
listOfStocks = []
listOfStocksOriginal = []
url = 'https://www.investors.com/etfs-and-funds/mutual-funds/best-mutual-funds-beating-sp-500-over-last-1-3-5-10-years/'
headers = {
'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.109 Safari/537.36'}
cprint('Get: ' + url, 'white', attrs=['dark'])
with Halo(spinner='dots'):
f = requests.get(url, headers=headers)
Functions.fromCache(f)
raw_html = f.text
soup = BeautifulSoup(raw_html, 'html.parser')
file = open('investors-stocks.txt', 'w')
r = soup.find_all('td')
for k in r:
t = k.text.strip()
if len(t) == 5 and Functions.strintIsFloat(t) is False:
if t not in listOfStocksOriginal or listOfStocksOriginal == []:
listOfStocksOriginal.append(t)
print(t, end=' ')
listOfStocks.append(k.text.strip())
file.write(str(k.text.strip()) + '\n')
file.close()
for i in range(0, len(listOfStocks), 1):
stockName = listOfStocks[i].upper()
listOfStocks[i] = Stock()
listOfStocks[i].setName(stockName)
print('\n' + str(len(listOfStocks)) + ' mutual funds total')
elif method == 7:
listOfStocks = []
url = 'https://finance.yahoo.com/screener/predefined/top_mutual_funds/'
headers = {
'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.109 Safari/537.36'}
cprint('Get: ' + url, 'white', attrs=['dark'])
with Halo(spinner='dots'):
f = requests.get(url, headers=headers)
Functions.fromCache(f)
raw_html = f.text
soup = BeautifulSoup(raw_html, 'html.parser')
file = open('yahoo-stocks.txt', 'w')
r = soup.find_all('a', attrs={'class': 'Fw(600)'})
for k in r:
t = k.text.strip()
if len(t) == 5 and t == t.upper():
print(t, end=' ')
listOfStocks.append(k.text.strip())
file.write(str(k.text.strip()) + '\n')
file.close()
for i in range(0, len(listOfStocks), 1):
stockName = listOfStocks[i].upper()
listOfStocks[i] = Stock()
listOfStocks[i].setName(stockName)
print('\n' + str(len(listOfStocks)) + ' mutual funds total')
if len(listOfStocks) < 2:
print('Please choose another method')
else:
@ -1089,12 +1154,12 @@ def sendAsync(url):
def timeFrameInit():
isInteger = False
while isInteger == False:
while isInteger is False:
print(
'\nPlease enter the time frame in months (<60 months recommended):', end='')
temp = input(' ')
isInteger = Functions.stringIsInt(temp)
if isInteger == True:
if isInteger is True:
if int(temp) > 1 and int(temp) < 1000:
months = int(temp)
else:
@ -1111,7 +1176,11 @@ def dataMain(listOfStocks):
i = 0
while i < len(listOfStocks):
datesAndCloseList = Stock.datesAndClose(listOfStocks[i])
try:
datesAndCloseList = Stock.datesAndClose(listOfStocks[i])
except:
print('Error retrieving data')
datesAndCloseList = 'N/A'
if datesAndCloseList == 'N/A':
del listOfStocks[i]
if len(listOfStocks) == 0:
@ -1229,7 +1298,7 @@ def returnMain(benchmark, listOfStocks):
cprint('\nNumber of stocks from original list that fit time frame: ' +
str(len(listOfStocks)), 'green')
if len(listOfStocks) < 2:
#print('Cannot proceed to the next step. Exiting program.')
# print('Cannot proceed to the next step. Exiting program.')
cprint('Cannot proceed to the next step. Exiting program.',
'white', 'on_red')
exit()
@ -1239,7 +1308,7 @@ def outlierChoice():
print('\nWould you like to remove indicator outliers?')
print('1. Yes\n2. No')
found = False
while found == False:
while found is False:
outlierChoice = str(input('Choice: '))
if Functions.stringIsInt(outlierChoice):
if int(outlierChoice) == 1:
@ -1265,12 +1334,12 @@ def indicatorInit():
print('List of indicators:')
for i in range(0, len(listOfIndicators), 1):
print(str(i + 1) + '. ' + listOfIndicators[i])
while indicatorFound == False:
while indicatorFound is False:
indicator = str(input('Choose an indicator from the list: '))
# indicator = 'expense ratio' # TESTING
if Functions.stringIsInt(indicator) == True:
if Functions.stringIsInt(indicator) is True:
if int(indicator) <= 4 and int(indicator) > 0:
indicator = listOfIndicators[int(indicator)-1]
indicatorFound = True
@ -1284,7 +1353,7 @@ def indicatorInit():
indicatorFound = True
break
if indicatorFound == False:
if indicatorFound is False:
print('Please choose an indicator from the list\n')
return indicator
@ -1374,7 +1443,7 @@ def plot_regression_line(x, y, b, i):
plt.pause(1)
sys.stdout.flush()
sys.stdout.write(
' \r')
' \r')
sys.stdout.flush()
plt.close()
@ -1382,10 +1451,10 @@ def plot_regression_line(x, y, b, i):
def persistenceTimeFrame():
print('\nTime frame you chose was', Stock.timeFrame, 'months')
persTimeFrameFound = False
while persTimeFrameFound == False:
while persTimeFrameFound is False:
persistenceTimeFrame = str(
input('Please choose how many months to measure persistence: '))
if Functions.stringIsInt(persistenceTimeFrame) == True:
if Functions.stringIsInt(persistenceTimeFrame) is True:
if int(persistenceTimeFrame) > 0 and int(persistenceTimeFrame) < Stock.timeFrame - 1:
persistenceTimeFrame = int(persistenceTimeFrame)
persTimeFrameFound = True
@ -1409,8 +1478,12 @@ def indicatorMain(listOfStocks):
listOfStocks[i].indicatorValue = Stock.calcPersistence(
listOfStocks[i])
else:
listOfStocks[i].indicatorValue = Stock.scrapeYahooFinance(
listOfStocks[i])
try:
listOfStocks[i].indicatorValue = Stock.scrapeYahooFinance(
listOfStocks[i])
except:
print('Error retrieving indicator data')
listOfStocks[i].indicatorValue = 'N/A'
print('')
if listOfStocks[i].indicatorValue == 'N/A':
@ -1428,11 +1501,11 @@ def indicatorMain(listOfStocks):
listOfStocksIndicatorValues.append(listOfStocks[i].indicatorValue)
# Remove outliers
if Stock.removeOutliers == True:
if Stock.removeOutliers is True:
cprint('\nRemoving outliers\n', 'white', attrs=['underline'])
temp = Functions.removeOutliers(listOfStocksIndicatorValues)
if temp[0] == listOfStocksIndicatorValues:
print('No outliers\n')
print('No indicator outliers\n')
else:
print('First quartile:', temp[2], ', Median:', temp[3],
', Third quartile:', temp[4], 'Interquartile range:', temp[5])
@ -1502,12 +1575,12 @@ def indicatorMain(listOfStocks):
def checkConfig(fileName):
if Functions.fileExists(fileName) == False:
if Functions.fileExists(fileName) is False:
return 'N/A'
file = open(fileName, 'r')
n = file.read()
file.close()
if Functions.validateJson(n) == False:
if Functions.validateJson(n) is False:
print('Config file is not valid')
return 'N/A'
t = json.loads(n)
@ -1516,12 +1589,14 @@ def checkConfig(fileName):
def main():
# Check config file for errors and if not, then use values
'''
Check config file for errors and if not, then use values
#! Only use this if you know it is exactly correct. I haven't spent much time debugging this
'''
Stock.config = checkConfig('config.json')
# Check if matplotlib is installed
if Functions.checkPackage('matplotlib') == False:
if Functions.checkPackage('matplotlib') is False:
print(
'matplotlib is not installed. This is required for plotting linear regression')
@ -1545,6 +1620,7 @@ def main():
exit()
else:
Functions.getJoke()
# Functions.getWeather()
# Choose benchmark and makes it class Stock
benchmark = benchmarkInit()
@ -1567,7 +1643,7 @@ def main():
# Choose whether to remove outliers or not
Stock.removeOutliers = outlierChoice()
else:
if Stock.config['Check Packages'] != False:
if Stock.config['Check Packages'] is not False:
packagesInstalled = Functions.checkPackages(
['numpy', 'requests', 'bs4', 'requests_cache', 'halo'])
if not packagesInstalled:
@ -1575,16 +1651,16 @@ def main():
else:
print('All required packages are installed')
if Stock.config['Check Python Version'] != False:
if Stock.config['Check Python Version'] is not False:
pythonVersionGood = Functions.checkPythonVersion()
if not pythonVersionGood:
exit()
if Stock.config['Check Internet Connection'] != False:
if Stock.config['Check Internet Connection'] is not False:
internetConnection = Functions.isConnected()
if not internetConnection:
exit()
if Stock.config['Get Joke'] != False:
if Stock.config['Get Joke'] is not False:
Functions.getJoke()
benchmarksTicker = ['SPY', 'DJIA', 'VTHR', 'EFT']
@ -1615,7 +1691,7 @@ def main():
Stock.persTimeFrame = persistenceTimeFrame()
# Choose whether to remove outliers or not
if Stock.config['Remove Outliers'] != False:
if Stock.config['Remove Outliers'] is not False:
Stock.removeOutliers = True
else:
Stock.removeOutliers = outlierChoice()
@ -1640,3 +1716,20 @@ def main():
if __name__ == "__main__":
main()
'''
Copyright (C) 2019 Andrew Dinh
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>.
'''