Income Streams vs. Chunks
When Custom Visuals started in 1995 (as Custom Data Visualizations), we were working on projects for individual companies with little emphasis on long-term projects utilizing our own servers. That all changed when an environmental company requested a web scraping application for a budget estimation project.
The project started off as a simple request to scrape a few dozen pages from a site and create PDF files for printing. As we discussed the details during an early meeting, I indicated I could programmatically pick out the information the client wanted from the website and drop it into a database for a summary report. Until this point, they had been hiring workers from a temp agency to navigate to the web pages, then save and/or print the page for the full-time workers to review.
After a couple days, I provided a sample report based on a small set of web pages for the client to review. That was a turning point in the project. What started as something that could be done in a week and provide a nice chunk of income, ended up as multiple enhancements over several years, with a high point of navigating more than 10,000 pages daily with 100’s of database-driven reports being sent to 40 or 50 employees across multiple divisions. That chunk of income turned into a steady stream of income as new features and requirements were added over the years.
Along the way, we upgraded to larger disks on the server to accommodate the data being retrieved. The database storage needed for reports was increasing rapidly as early daily reports now spanned longer time periods and included additional parameters. Then we upgraded the server for more speed and kicked in more memory, as well. And we’ve done the same thing a few more times since then, too.
What started off as a quick little project turned into a nice monthly source of income since we were navigating, downloading, analyzing, reporting and storing 100s of GBs of data and using significant processing time to do so. Eventually, it made sense to waive the development costs and just add new features into a monthly fee.
During the middle of this project, we decided this was the best approach for our small business to take. Rather than the feast or famine mode that goes along with projects in chunks, we focused on long-term projects that ran on our own servers to generate streams. We still happily pick up the occasional chunk, but we keep an eye out for steady streams.