Stock Data Miner

SUMMARY OF SOFTWARE This project was started in python, you can finish the current project OR you can start from scratch using any coding language that will perform the tasks needed. Please let me know your plan, price and time for completion of the project with your bid. download the current unfinished python source here [login to view URL] This is a Python program that mines stock information from websites. It downloads the data for stocks, and then computes additional data based on the mined data. Once all downloading and calculations are complete, reports can be exported to .XLS and .CSV files. The .XLS file is a human-readable format used to aid in choosing stocks to buy/sell/hold. The .CSV files are also human-readable, but are meant to be used as the backend data for a website. There are .PHP files on the website that can read, parse, and display the data contained in the .CSV files. A lot of the data displayed on the various tabs is stored in .DAT files in the "current directory" that the program runs in. Some of the .DAT files cache the data downloaded from web pages. Other .DAT files store user input, so that the user does not have to enter it each time the program is run. When downloading web pages, the web pages are cached in a folder named 'cache' within the "current directory" that the program runs in. This is useful mainly for testing the program. Instead of waiting for hours to re-download all data, the data can simply be read from disk when testing. This can save the developer a lot of time, although the user of the software may not find this functionality as useful. This software attempts to be "polite" to the MSN website. It is possible to speed up the downloading of web pages, however we would not want the website to reject us for unusually fast queries. The length of time that this program takes to run is not really an issue, considering how few times per year it is intended to run. IMPORTANT NOTE: It is recommended that the software be deployed to the user with the .DAT files that are used to store user input, or else it is recommended that the user copy in his .DAT files from the old software distribution to the new one. ## Deliverables SUMMARY OF SOFTWARE This project was started in python, you can finish the current project OR you can start from scratch using any coding language that will perform the tasks needed. Please let me know your plan, price and time for completion of the project with your bid. download the current unfinished python source here [login to view URL] This is a Python program that mines stock information from websites. It downloads the data for stocks, and then computes additional data based on the mined data. Once all downloading and calculations are complete, reports can be exported to .XLS and .CSV files. The .XLS file is a human-readable format used to aid in choosing stocks to buy/sell/hold. The .CSV files are also human-readable, but are meant to be used as the backend data for a website. There are .PHP files on the website that can read, parse, and display the data contained in the .CSV files. A lot of the data displayed on the various tabs is stored in .DAT files in the "current directory" that the program runs in. Some of the .DAT files cache the data downloaded from web pages. Other .DAT files store user input, so that the user does not have to enter it each time the program is run. When downloading web pages, the web pages are cached in a folder named 'cache' within the "current directory" that the program runs in. This is useful mainly for testing the program. Instead of waiting for hours to re-download all data, the data can simply be read from disk when testing. This can save the developer a lot of time, although the user of the software may not find this functionality as useful. This software attempts to be "polite" to the MSN website. It is possible to speed up the downloading of web pages, however we would not want the website to reject us for unusually fast queries. The length of time that this program takes to run is not really an issue, considering how few times per year it is intended to run. IMPORTANT NOTE: It is recommended that the software be deployed to the user with the .DAT files that are used to store user input, or else it is recommended that the user copy in his .DAT files from the old software distribution to the new one. The software can be deployed as a Windows .EXE file. -------------------------------------------------------------------- OPERATING SYSTEM DEPENDENCIES This program is designed to run on Windows. It will run on other platforms such Linux and MacOS, but the Excel speadsheet export functionality will not work, as it is Microsoft-specific. This software has been tested on Microsoft Windows XP and Mircrosoft Windows 7, as well as various versions of Ubuntu Linux. -------------------------------------------------------------------- SOFTWARE CODE DETAILS [login to view URL] - The main entry point of the software. This contains the code for the GUI, as well as most of the business logic. [login to view URL] - Utility functions for catching and managing errors and unexpected events in the software. [login to view URL] - Utility functions to format and error-check data, utilties for downloading files from the internet. [login to view URL] - Utility for converting currencies. [login to view URL] - Not part of the program. This script is used to build a Windows executable (.EXE) file. Current Python line count: 1577 lines of code. (This excludes comments and whitespace) Based on the line count, this software can be classified as "small" in scope. However, please note that this program receives a lot of its input from web pages, and we do not have control over their content. This means that small changes to the software require good testing. Good testing is also important because of the target use of the program: making decisions about money. Some of the modules in this software contain test code. They should be marked. Some test code is embedded within the production code (and commented as such) to enable easy testing. There are other sections of code that are completely dedicated to testing. These should be named or commented as tested code. IMPORTANT NOTE: If the MSN website changes the format of their web pages, the data miners in this code may need to be changed. When mining data, we are looking for certain patterns in the HTML. Even small changes to the HTML on the website could have large impacts to this program's ability to locate where the data is. ------------------------------------------------------------------------------- SOFTWARE PACKAGE DEPENDENCIES This program is written in Python. The program is compatible with Python 2.5.4, and should be compatible with later 2.x versions. [[login to view URL]] The Python program depends on the following libraries to run: wxPython 2.8 or greater [[login to view URL]] The Python progam depends on the following for Excel spreadsheet functionality to work: PyWin32 library [[login to view URL]] Microsoft Excel must be installed. In addition, to compile to executable, the following library is needed: py2exe [[login to view URL]]