Added options to restart program and plot indicator linear regression

Added link to the essay that was the basis of this project

CONTRIBUTING.md

@@ -30,7 +30,7 @@ To develop on fund-indicators you'll probably want to:
 1. Read the [code of conduct](https://github.com/andrewkdinh/fund-indicators/blob/master/CODE-OF-CONDUCT.md)
 2. Install normally, as documented in the [README](https://github.com/andrewkdinh/fund-indicators#quickstart)
 3. Read the [documentation](https://github.com/andrewkdinh/fund-indicators/wiki) (although it's not much)
-4. Read the section below to understand the ways you could best help this project
+4. Read the section above to understand the ways you could best help this project
 
 ## Questions?
 

Functions.py

@@ -79,7 +79,7 @@ def getJoke():
                    'User-Agent': 'fund-indicators (https://github.com/andrewkdinh/fund-indicators)'}
         url = 'https://icanhazdadjoke.com'
 
-        cprint('Get: ' + url, 'white', attrs=['dark'])
+        cprint('Get:' + url, 'white', attrs=['dark'])
         with Halo(spinner='dots'):
             f = requests.get(url,
                              headers=headers).json()
@@ -209,13 +209,47 @@ def getWeather():
     with requests_cache.disabled():
         url = 'https://wttr.in?format=3'
 
-        cprint('Get: ' + url, 'white', attrs=['dark'])
+        cprint('Get:' + url, 'white', attrs=['dark'])
         with Halo(spinner='dots'):
             f = requests.get(url)
         print('')
         print(colored('Current weather in ' + f.text, 'green'), end='')
 
 
+def detectDisplay():
+    import os
+    try:
+        t = os.environ["DISPLAY"]
+    except KeyError:
+        return False
+    if t == ':0.0':
+        return True
+    else:
+        return False
+
+
+def trueOrFalse():
+    found = False
+    print('[1] Yes\n[2] No')
+    while found is False:
+        answer = str(input('Answer: '))
+        if stringIsInt(answer) is True:
+            temp = int(answer)
+            if temp == 1:
+                return True
+            elif temp == 2:
+                return False
+            else:
+                print('Please choose either 1 or 2')
+                pass
+        elif answer.lower() == 'yes':
+            return True
+        elif answer.lower() == 'no':
+            return False
+        else:
+            print('Please either choose a number or type an answer')
+
+
 def main():
     exit()
 

README.md

@@ -6,16 +6,10 @@
 ![](https://img.shields.io/github/languages/code-size/andrewkdinh/fund-indicators.svg)
 [![CII Best Practices](https://bestpractices.coreinfrastructure.org/projects/2667/badge)](https://bestpractices.coreinfrastructure.org/projects/2667)
 
-A project to determine relationships between mutual fund performance and different factors.
+fund-indicators is a cross-platform Python application that allows users to easily find relationships between various attributes of mutual funds and previous performance. This project is based on research from [*Performance Indicators of Mutual Funds*](https://nextcloud.andrewkdinh.com/s/xNgGQ4nPNkSqJti).
 
 [![asciicast demo](https://asciinema.org/a/jLmZapnMFGCRiiSUITY21erLW.svg)](https://asciinema.org/a/jLmZapnMFGCRiiSUITY21erLW?autoplay=1&preload=1)
 
-Calculates relationships between: Previous performance, Alpha, Sharpe Ratio, Sortino Ratio
-
-and Expense ratios, Turnover, Market Capitalization (Asset Size), Persistence
-
-Give it a try at [repl.run](https://fund-indicators.andrewkdinh.repl.run) or [repl.it](https://repl.it/@andrewkdinh/fund-indicators).
-
 ## Key Features
 
 - 100% automated
@@ -26,9 +20,14 @@ Give it a try at [repl.run](https://fund-indicators.andrewkdinh.repl.run) or [re
 - Optional graphs to easily visualize linear regression results
 - A new joke every time
 - Cross-platform (tested on Windows and Linux)
+- Simple to use
 
 ## Quickstart
 
+Give it a try at [repl.run](https://fund-indicators.andrewkdinh.repl.run) or [repl.it](https://repl.it/@andrewkdinh/fund-indicators).
+
+If you would like to clone to your own machine:
+
 ```shell
 git clone https://github.com/andrewkdinh/fund-indicators.git && cd fund-indicators
 pip install -r requirements.txt
@@ -38,7 +37,12 @@ python main.py
 - Common mutual funds are listed in `stocks.txt`
 - Configure and rename `config.example.json` to `config.json` if you would like to skip beginning questions (only for advanced users)
 
-### Contributing
+## Planned Features
+
+- Graphical user interface (GUI)
+- Multithreading/asynchronous requests
+
+## Contributing
 
 Want to help? Great! Check out the [CONTRIBUTING.md](https://github.com/andrewkdinh/fund-indicators/blob/master/CONTRIBUTING.md) file!
 
@@ -46,7 +50,7 @@ Want to help? Great! Check out the [CONTRIBUTING.md](https://github.com/andrewkd
 
 This project utilizes a wide variety of open-source projects:
 
-- [NumPy](https://github.com/numpy/numpy), [Termcolor](https://github.com/hfeeki/termcolor), [Beautiful Soup](https://launchpad.net/beautifulsoup), [yahoofinancials](https://github.com/JECSand/yahoofinancials), [requests-cache](https://github.com/reclosedev/requests-cache), [halo](https://github.com/manrajgrover/halo), [matplotlib](https://github.com/matplotlib/matplotlib), [asciinema](https://github.com/asciinema/asciinema)
+- [NumPy](https://github.com/numpy/numpy), [Termcolor](https://github.com/hfeeki/termcolor), [Beautiful Soup](https://launchpad.net/beautifulsoup), [yahoofinancials](https://github.com/JECSand/yahoofinancials), [requests-cache](https://github.com/reclosedev/requests-cache), [halo](https://github.com/manrajgrover/halo), [matplotlib](https://github.com/matplotlib/matplotlib), [asciinema](https://github.com/asciinema/asciinema), [Core Infrastructure Initiative Best Practices Badge](https://github.com/coreinfrastructure/best-practices-badge)
 
 And thank you to those that have helped me with the idea and product:
 

main.py

@@ -19,7 +19,10 @@ from bs4 import BeautifulSoup
 import numpy as np
 
 # OPTIONAL
-# import matplotlib.pyplot as plt
+try:
+    import matplotlib.pyplot as plt
+except:
+    pass
 from halo import Halo
 
 # FOR ASYNC
@@ -79,6 +82,7 @@ class Stock:
     # CONFIG
     removeOutliers = True
     sourceList = ['Yahoo', 'Alpha Vantage', 'IEX', 'Tiingo']
+    plotIndicatorRegression = False
     config = 'N/A'
 
     # BENCHMARK VALUES
@@ -135,7 +139,7 @@ class Stock:
         url = ''.join(
             ('https://api.iextrading.com/1.0/stock/', self.name, '/chart/5y'))
         # link = "https://api.iextrading.com/1.0/stock/spy/chart/5y"
-        cprint("Get: " + url, 'white', attrs=['dark'])
+        cprint("Get:" + url, 'white', attrs=['dark'])
         with Halo(spinner='dots'):
             f = requests.get(url)
         Functions.fromCache(f)
@@ -174,7 +178,7 @@ class Stock:
                        self.name, '&outputsize=full&apikey=', apiAV))
         # https://www.alphavantage.co/query?function=TIME_SERIES_DAILY_ADJUSTED&symbol=MSFT&outputsize=full&apikey=demo
 
-        cprint("Get: " + url, 'white', attrs=['dark'])
+        cprint("Get:" + url, 'white', attrs=['dark'])
         with Halo(spinner='dots'):
             f = requests.get(url)
         Functions.fromCache(f)
@@ -211,7 +215,7 @@ class Stock:
             'Authorization': token
         }
         url = ''.join(('https://api.tiingo.com/tiingo/daily/', self.name))
-        cprint("Get: " + url, 'white', attrs=['dark'])
+        cprint("Get:" + url, 'white', attrs=['dark'])
         with Halo(spinner='dots'):
             f = requests.get(url, headers=headers)
         Functions.fromCache(f)
@@ -233,7 +237,7 @@ class Stock:
         url2 = ''.join((url, '/prices?startDate=',
                         firstDate, '&endDate=', lastDate))
         # https://api.tiingo.com/tiingo/daily/<ticker>/prices?startDate=2012-1-1&endDate=2016-1-1
-        cprint("\nGet: " + url2 + '\n', 'white', attrs=['dark'])
+        cprint("\nGet:" + url2 + '\n', 'white', attrs=['dark'])
         with Halo(spinner='dots'):
             requestResponse2 = requests.get(url2, headers=headers)
         Functions.fromCache(requestResponse2)
@@ -261,7 +265,7 @@ class Stock:
     def Yahoo(self):
         url = ''.join(('https://finance.yahoo.com/quote/',
                        self.name, '?p=', self.name))
-        cprint('Get: ' + url, 'white', attrs=['dark'])
+        cprint('Get:' + url, 'white', attrs=['dark'])
         with Halo(spinner='dots'):
             t = requests.get(url)
         if t.history:
@@ -563,7 +567,7 @@ class Stock:
         # Determine if ETF, Mutual fund, or stock
         url = ''.join(('https://finance.yahoo.com/quote/',
                        self.name, '?p=', self.name))
-        cprint('Get: ' + url, 'white', attrs=['dark'])
+        cprint('Get:' + url, 'white', attrs=['dark'])
         with Halo(spinner='dots'):
             t = requests.get(url)
         Functions.fromCache(t)
@@ -575,7 +579,7 @@ class Stock:
 
         stockType = ''
         url2 = ''.join(('https://finance.yahoo.com/lookup?s=', self.name))
-        cprint('Get: ' + url2, 'white', attrs=['dark'])
+        cprint('Get:' + url2, 'white', attrs=['dark'])
         with Halo(spinner='dots'):
             x = requests.get(url2)
             raw_html = x.text
@@ -733,7 +737,7 @@ class Stock:
                 url = ''.join(('https://finance.yahoo.com/quote/',
                                self.name, '/profile?p=', self.name))
                 # https://finance.yahoo.com/quote/SPY/profile?p=SPY
-                cprint('Get: ' + url, 'white', attrs=['dark'])
+                cprint('Get:' + url, 'white', attrs=['dark'])
                 with Halo(spinner='dots'):
                     raw_html = requests.get(url).text
                 soup = BeautifulSoup(raw_html, 'html.parser')
@@ -811,7 +815,7 @@ def benchmarkInit():
     benchmarksTicker = ['SPY', 'DJIA', 'VTHR', 'EFT']
     print('\nList of benchmarks:')
     for i in range(0, len(benchmarks), 1):
-        print(str(i+1) + '. ' +
+        print('[' + str(i+1) + '] ' +
               benchmarks[i] + ' (' + benchmarksTicker[i] + ')')
     while benchmarkTicker == '':
 
@@ -862,7 +866,7 @@ def stocksInit():
                    'Yahoo top mutual funds (25)']
 
         for i in range(0, len(methods), 1):
-            print(str(i+1) + '. ' + methods[i])
+            print('[' + str(i+1) + '] ' + methods[i])
         while method == 0 or method > len(methods):
             method = str(input('Which method? '))
             if Functions.stringIsInt(method) is True:
@@ -877,9 +881,9 @@ def stocksInit():
         if method == 1:
             defaultFiles = ['.gitignore', 'LICENSE', 'main.py', 'Functions.py',
                             'README.md', 'requirements.txt', 'cache.sqlite',
-                            'yahoofinancials.py', 'termcolor.py',
+                            'config.json', 'CONTRIBUTING.md',
-                            'README.html', 'config.json',
+                            'config.example.json', '_test_runner.py',
-                            'config.example.json', '_test_runner.py']
+                            'CODE-OF-CONDUCT.md']
             # Added by repl.it for whatever reason
             stocksFound = False
             print('Files in current directory (without default files): ')
@@ -890,7 +894,7 @@ def stocksInit():
                     listOfFiles.append(files)
             for i in range(0, len(listOfFiles), 1):
                 if listOfFiles[i][0] != '.':
-                    print(str(i+1) + '. ' + listOfFiles[i])
+                    print('[' + str(i+1) + '] ' + listOfFiles[i])
             while stocksFound is False:
                 fileName = str(input('What is the file number/name? '))
                 if Functions.stringIsInt(fileName) is True:
@@ -921,7 +925,7 @@ def stocksInit():
         elif method == 2:
             isInteger = False
             while isInteger is False:
-                temp = input('\nNumber of stocks to analyze (2 minimum): ')
+                temp = input('Number of stocks to analyze (2 minimum): ')
                 isInteger = Functions.stringIsInt(temp)
                 if isInteger is True:
                     if int(temp) >= 2:
@@ -951,7 +955,7 @@ def stocksInit():
             url = 'https://www.kiplinger.com/tool/investing/T041-S001-top-performing-mutual-funds/index.php'
             headers = {
                 'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.109 Safari/537.36'}
-            cprint('Get: ' + url, 'white', attrs=['dark'])
+            cprint('Get:' + url, 'white', attrs=['dark'])
             with Halo(spinner='dots'):
                 f = requests.get(url, headers=headers)
             Functions.fromCache(f)
@@ -978,7 +982,7 @@ def stocksInit():
             url = 'https://www.thestreet.com/topic/21421/top-rated-mutual-funds.html'
             headers = {
                 'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.109 Safari/537.36'}
-            cprint('Get: ' + url, 'white', attrs=['dark'])
+            cprint('Get:' + url, 'white', attrs=['dark'])
             with Halo(spinner='dots'):
                 f = requests.get(url, headers=headers)
             Functions.fromCache(f)
@@ -1008,7 +1012,7 @@ def stocksInit():
             url = 'http://money.com/money/4616747/best-mutual-funds-etfs-money-50/'
             headers = {
                 'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.109 Safari/537.36'}
-            cprint('Get: ' + url, 'white', attrs=['dark'])
+            cprint('Get:' + url, 'white', attrs=['dark'])
             with Halo(spinner='dots'):
                 f = requests.get(url, headers=headers)
             Functions.fromCache(f)
@@ -1041,7 +1045,7 @@ def stocksInit():
             url = 'https://www.investors.com/etfs-and-funds/mutual-funds/best-mutual-funds-beating-sp-500-over-last-1-3-5-10-years/'
             headers = {
                 'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.109 Safari/537.36'}
-            cprint('Get: ' + url, 'white', attrs=['dark'])
+            cprint('Get:' + url, 'white', attrs=['dark'])
             with Halo(spinner='dots'):
                 f = requests.get(url, headers=headers)
             Functions.fromCache(f)
@@ -1055,6 +1059,7 @@ def stocksInit():
                 t = k.text.strip()
                 if len(t) == 5 and Functions.strintIsFloat(t) is False:
                     if t not in listOfStocksOriginal or listOfStocksOriginal == []:
+                        if t[-1] != '%':
                             listOfStocksOriginal.append(t)
                             print(t, end=' ')
                             listOfStocks.append(k.text.strip())
@@ -1073,7 +1078,7 @@ def stocksInit():
             url = 'https://finance.yahoo.com/screener/predefined/top_mutual_funds/'
             headers = {
                 'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.109 Safari/537.36'}
-            cprint('Get: ' + url, 'white', attrs=['dark'])
+            cprint('Get:' + url, 'white', attrs=['dark'])
             with Halo(spinner='dots'):
                 f = requests.get(url, headers=headers)
             Functions.fromCache(f)
@@ -1147,7 +1152,7 @@ def asyncData(benchmark, listOfStocks):
 
 def sendAsync(url):
     time.sleep(random.randrange(0, 2))
-    cprint('Get: ' + url, 'white', attrs=['dark'])
+    cprint('Get:' + url, 'white', attrs=['dark'])
     requests.get(url)
     return
 
@@ -1156,7 +1161,7 @@ def timeFrameInit():
     isInteger = False
     while isInteger is False:
         print(
-            '\nPlease enter the time frame in months (<60 months recommended):', end='')
+            '\nPlease enter the time frame in months (<60 recommended):', end='')
         temp = input(' ')
         isInteger = Functions.stringIsInt(temp)
         if isInteger is True:
@@ -1206,7 +1211,7 @@ def riskFreeRate():
         ('https://www.quandl.com/api/v3/datasets/USTREASURY/LONGTERMRATES.json?api_key=', apiQuandl))
     # https://www.quandl.com/api/v3/datasets/USTREASURY/LONGTERMRATES.json?api_key=KUh3U3hxke9tCimjhWEF
 
-    cprint('\nGet: ' + url, 'white', attrs=['dark'])
+    cprint('\nGet:' + url, 'white', attrs=['dark'])
     with Halo(spinner='dots'):
         f = requests.get(url)
     Functions.fromCache(f)
@@ -1290,12 +1295,12 @@ def returnMain(benchmark, listOfStocks):
             listOfStocks[i].sharpe = Stock.calcSharpe(listOfStocks[i])
             listOfStocks[i].sortino = Stock.calcSortino(listOfStocks[i])
             listOfStocks[i].treynor = Stock.calcTreynor(listOfStocks[i])
-            listOfStocks[i].linearRegression = Stock.calcLinearRegression(
+            # listOfStocks[i].linearRegression = Stock.calcLinearRegression(
-                listOfStocks[i])
+            #     listOfStocks[i])
 
             i += 1
 
-    cprint('\nNumber of stocks from original list that fit time frame: ' +
+    cprint('\nNumber of stocks that fit time frame: ' +
            str(len(listOfStocks)), 'green')
     if len(listOfStocks) < 2:
         # print('Cannot proceed to the next step. Exiting program.')
@@ -1306,7 +1311,7 @@ def returnMain(benchmark, listOfStocks):
 
 def outlierChoice():
     print('\nWould you like to remove indicator outliers?')
-    print('1. Yes\n2. No')
+    print('[1] Yes\n[2] No')
     found = False
     while found is False:
         outlierChoice = str(input('Choice: '))
@@ -1333,7 +1338,7 @@ def indicatorInit():
     print('\n', end='')
     print('List of indicators:')
     for i in range(0, len(listOfIndicators), 1):
-        print(str(i + 1) + '. ' + listOfIndicators[i])
+        print('[' + str(i + 1) + '] ' + listOfIndicators[i])
     while indicatorFound is False:
         indicator = str(input('Choose an indicator from the list: '))
 
@@ -1396,7 +1401,8 @@ def calcIndicatorRegression(listOfIndicatorValues, listOfReturns):
         regression.append(b[1])
         regressionList.append(regression)
 
-        # plot_regression_line(x, y, b, i)
+        if Stock.plotIndicatorRegression is True:
+            plot_regression_line(x, y, b, i)
 
     return regressionList
 
@@ -1588,20 +1594,42 @@ def checkConfig(fileName):
     return r
 
 
+def continueProgram():
+    found = False
+    print('Would you like to rerun the program?')
+    return Functions.trueOrFalse()
+
+
+def plotIndicatorRegression():
+    if Functions.detectDisplay() is True:
+        if Functions.checkPackage('matplotlib') is False:
+            print(
+                'matplotlib is not installed. \nIf you would like to install' +
+                ' it (and have a display), run `pip install matplotlib`')
+            return False
+        else:
+            print('\nWould you like to plot indicator linear regression '
+                  'results?')
+            plotLinear = Functions.trueOrFalse()
+            if plotLinear is True:
+                return True
+            else:
+                return False
+
+
 def main():
     '''
     Check config file for errors and if not, then use values
-    #! Only use this if you know it is exactly correct. I haven't spent much time debugging this
+    #! Only use this if you know it is exactly correct. I haven't spent much
+    #! time debugging this
     '''
     Stock.config = checkConfig('config.json')
 
-    # Check if matplotlib is installed
+    runningProgram = True
-    if Functions.checkPackage('matplotlib') is False:
+    while runningProgram is True:
-        print(
-            'matplotlib is not installed. This is required for plotting linear regression')
 
-    # Check that all required packages are installed
         if Stock.config == 'N/A':
+            # Check that all required packages are installed
             packagesInstalled = Functions.checkPackages(
                 ['numpy', 'requests', 'bs4', 'requests_cache', 'halo'])
             if not packagesInstalled:
@@ -1632,7 +1660,7 @@ def main():
 
             # Determine time frame (Years)
             timeFrame = timeFrameInit()
-        Stock.timeFrame = timeFrame  # Needs to be a global variable for all stocks
+            Stock.timeFrame = timeFrame
 
             # Choose indicator
             Stock.indicator = indicatorInit()
@@ -1642,6 +1670,11 @@ def main():
 
             # Choose whether to remove outliers or not
             Stock.removeOutliers = outlierChoice()
+
+            # Check if matplotlib is installed and if so, ask user if
+            # they want to plot
+            Stock.plotIndicatorRegression = plotIndicatorRegression()
+
         else:
             if Stock.config['Check Packages'] is not False:
                 packagesInstalled = Functions.checkPackages(
@@ -1710,7 +1743,11 @@ def main():
         # Choose indicator and calculate correlation with indicator
         indicatorMain(listOfStocks)
 
+        # Decide if running program again
         print('')
+        runningProgram = continueProgram()
+        print('')
+
     exit()