August 14 & 15, 2021
A Small Inspiration
I had a talk with someone who needed some advice about the next step he should take his data self-study. This person was feeling the pressure of family and time. He felt lost and was at the point of giving up. It was hard to tell his family that he was not getting a job as an engineer as he had studied for in University. He was changing to being a programmer working with data for which he didn’t have a degree.
I found him in our #66DaysofData Discord and reached out to him to get an idea what his situation was. Most of us on the server are in this same situation. As self-learners, we are usually isolated from others who share our interest in data and statistics. It’s hard to talk to others about it, so we need this community in order to motivate each other and maintain our accountability.
He seemed to have hit that wall that everyone comes to when they are laying out a plan of independent study. And that is the fear and anxiety that what you are doing isn’t getting you anywhere. Avoiding the negative aspects of his feelings, I asked him directly what projects and techniques had he found interesting.
I had him focus on the process of listening to small ideas and acting on them. If you want to scrape a website, go find one that you want to scrape. Don’t just scrape one without it meaning anything (or you could if you want). If a big project scares you, just work on the smaller techniques that will lead you to eventually realize a more complete project.
What really interests you? It doesn’t matter but it is important to answer.
You should always ask what would be interesting to do and then….do it! I find that we don’t often do the most interesting things. However, it is important to find some novelty like one of those small, funny, intuitive thoughts that occasionally come to us where we say…”I wonder what would happen if I did this…..”. Most of the time we stop ourselves and say, “No, that is silly or impossible and we need to be practical.”
With that in mind, I suggested that he find something outside of data and computers that interested him. He decided on football. I told him to do something with that. He chose to scrape a website for a NLP project to analyse the relation between sentiment of commentators and the performance of the players. This sounded like a compelling but sophisticated project that could easily motivate him. Yet, it could easily discourage him without a clear direction if any problems arise along the way. Many things could cause a project to fail. But, he went off to work with a little more focus.
Now….do it….or do something else….but do something! It doesn’t matter what…
When he came back a few hours later, he had scraped a site of housing rentals in India. It wasn’t the plan but I said “That’s great!”. Yet, the dataset was clean, large and in a place in the world that doesn’t feature prominently in the typical practice datasets. We have the Boston and Ames datasets from the states. One is a little outdated and simplified and the other is a little more formidable but realistic. So, this guy had pulled out a dataset from India that was sufficiently populated and sophisticated enough to be interesting. And, I said, “You should put that on Kaggle”. It was a small idea and he did it. Check it out here.
His dataset got some positive attention and he soon earned his first medal. That ignited his interest again. You see the point isn’t that you HAVE TO do the big project you had planned and are failing to complete, but that you listened to a small idea and did it. A lot of small ideas build to bigger ones. They interact and activate each other and a chain reaction will eventually take place that will give you a full project that you never imagined you could do. Now that he had this dataset, he worked on analyzing it and visualizing it and all the interesting things we do with data.
Don’t be surprised if things happen…or if they don’t
About three months later, he got his first internship. They told him that that dataset which he had created and published was an important factor in his selection for the opportunity. He called me and thanked me. All of this really impressed me.
Here, I am struggling to get a hiring recruiter to look my way. The time I spend everyday looking for a job, studying some stats and/or practicing my python skills is significant. I am working on a projects, attending some talks about job searching, having my dependencies break and solving that problem. Even writing this blog has become a weight that I have to lift.
Discouragement and energy drain are always imminent. To hear about such a simple thing which I suggested to someone who was in the same situation that is currently looming over me right now. My small suggestion to him was enough for him to receive a significant and important opportunity. This made me rethink how I am doing things.
So, I took my advice and immediately started a web scraping project. It wasn’t such a blazing success as when he did project. I realized that so much of my bootcamp had webscraping that it seemed that I needed to practice other skills. This made me neglect it for the following months.
Always Take Action
Upon returning to webscraping, two things occurred to me. First, my skill had gotten a little rusty. I found myself scouring the docs page, thinking to myself….”How do you do this again?”. Second, it is where many projects start and grow out of. Webscraping gives you the dataset that you can later apply to all sorts of analysis. It is a cornerstone for what we are doing because it also adds values and enrich other datasets.
I chose to scrape a job postings website in order to see how to incorporate a more automated application process. It takes so much time and if I could integrate some a pipeline of python automation for applying to jobs then I could at least practice an important skill while I apply! That is called stacking. I used to suggest to my ESL students that if they find they don’t have much time to commit to studying English that they combine or stack their tasks. They could help their children with English homework which would cover spending time with their children, ensuring they get their homework done, and reinforce some basic things they need to know.
So, you need to combine tasks when you take on more responsibilites. Listen to the small voice in your head that might help you make new serendipitous connections. This is how we overcome difficult problems and strengthen our creative reflex. Never be dogmatic or rigid. And don’t overcomplicate things…because they are complicated enough without your help. Practice the simplification of everything. You might be able to find innovative solutions….or simply easier ones.