Guilherme Pinheiro's Portfolio

Projects

A personal website to showcase my projects and experiences.

Algorithm that is segmented into 4 parts: Stock Analysis System, Recommendation Engine, Economic Factors Monitor and Identity Service.

Developed By: Me

Git

Github

Oracle

Python

SAA: Unlocking the Power of Value Investing

The project began with a strong desire to dive deeper into various tech stacks and explore existing software engineering patterns. Alongside this, my enduring love for data and creating algorithms that can truly make a difference fueled my motivation. A key personal goal has been to generate a passive income that surpasses my monthly earnings. Combining these three ambitions, I embarked on developing a Stock Picking Algorithm.

The journey was not without its challenges. Initially, I was faced with identifying the most suitable tech stack. Just two months ago, I decided to undertake a major refactor, segmenting the algorithm into four distinct systems:

🚦 Stock Analysis System: Leveraging principles from Value/Fundamental Investing and basic technical analysis inspired by various books and articles.

💡 Recommendation Engine: Utilizing my database to suggest which stocks to buy or monitor.

📊 Economic Factors Monitor: Keeping track of critical economic indicators.

🔐 Identity Service: Implementing a security layer for my services API's.

It is important to note that this algorithm currently operates automatically for US stocks only. I source data freely from the SEC and other APIs, with economic indicators being retrieved from the Bureau of Economic Analysis (BEA).

Remarkably, the system has been running for a year now, and I have managed to keep the total cost at ZERO euros. The portfolio's average profitability stands at around 18%, despite some initial missteps in stock selection. Today, I have much greater confidence in the algorithm's capabilities.

Some might wonder, why not simply invest in an S&P 500 fund? For me, the answer is clear: I wanted to learn how the great minds picked the best stocks in the past.

SAA: Data Acquisition

For the stock analysis algorithm, I only analyse financial data of US-listed companies. My goal is to continue having zero costs while ensuring data quality. Here's a brief overview of my approach and the tools I use.

🇺🇸 Why US Public Companies?

US public companies must disclose their financial statements publicly, providing a wealth of free, high-quality data. This makes the US market an ideal starting point for algorithm development.

💾 Data Sources and APIs

SEC API: The Securities and Exchange Commission offers a reliable API to access a wide range of financial statements and filings of public companies. It is the cornerstone of my data strategy.

BEA API: The Bureau of Economic Analysis API provides valuable economic data, complementing the financial data from the SEC.

Other APIs: I've also tested APIs like Alpha Spread, FRED, and unofficial Yahoo Finance APIs. However, these sources often have confusing data structures, requiring additional processing.

🔴 Challenges and Considerations

Data Consistency: Ensuring data from different sources is consistent and high-quality is challenging.

Cost Management: By using publicly available data, I have none costs.

API Limitations: Navigating rate limits and other constraints of each API is crucial for smooth data acquisition.

🌫 Future Directions

I plan to expand the algorithm's scope to include companies outside the US, identifying new data sources while maintaining a cost-effective approach. My next steps on this matter is to get data as reported. Data that was not processed by others.

SAA: Validating Data Updates and Managing API Rate Limits

🔍 Validating Data Updates

Check for Existing Data

First, I determine if the stock data already exists in my database.

If not, I request all necessary data from external APIs.

Verify Annual and Quarterly Data

For existing stocks, I check if the annual and quarterly data is up to date.

If discrepancies are found, I update the data accordingly.

Retrieve Real-Time Data

For real-time data, I always query external APIs to ensure the most current information.

🌐 Managing API Rate Limits

Understand Rate Limits

Familiarize yourself with each API's rate limits and constraints.

Monitor and Alert

I launch an exception before reaching the hitting limits. I created an exception called RateLimitError.

SAA: From AWS to Raspberry Pi to Oracle 🚀

Navigating the world of data storage has been quite the adventure for me! Here's a glimpse into my journey and the lessons I've learned along the way:

🌟 AWS Free Tier & PostgreSQL

I started my journey using AWS's free tier with PostgreSQL. It seemed like a perfect start until I discovered that AWS enabled diagnostics by default, which led to unexpected monthly charges. This experience taught me the importance of thoroughly understanding the cost implications of cloud services.

🔧 Building a Raspberry Pi Server with MariaDB

Determined to find a cost-effective solution, I decided to build my own server using a Raspberry Pi and MariaDB. This DIY approach was a fantastic learning experience, and I enjoyed the hands-on work. However, maintaining the server became increasingly time-consuming and tedious over the course of a year.

🧑‍💻 Transition to Oracle with PL SQL

Eventually, I transitioned to using Oracle with PL SQL. This shift came with its own set of challenges, primarily adapting my queries and algorithms to fit a new database system. However, it also provided a robust and reliable solution that required less day-to-day maintenance.

🎓 Lessons Learned

Cost Awareness: Always investigate the potential hidden costs of cloud services.

Hands-on Experience: Building and maintaining your own server is invaluable for learning but can be labor-intensive.

Flexibility: Be prepared to adapt and rewrite your queries and algorithms when switching databases.

Reliability: Sometimes, investing in a more robust, albeit complex, solution pays off in the long run.

Learn by listening, leave the reading for us.