I hate to be that kind of person, but if you want to work with SEC filings “at scale”, you should use the bulk download link that the SEC provides. And if you want to get large lists of filings, you should use the master index files that the SEC provides. This EFTS API thing is really only good for checking what was filed on the current day, before the master index for the day is generated.
Other than that, it’s a very nice job. Oh, and if the GitHub account owner reads this, I would like to point out that GitHub clearly identifies which school you go to, and your blog (linked in the project) says that you created this project because it would have cost $35k to get access to the dataset. Well, that’s wrong. Over at the business school within your fine university, tell them you’d like a WRDS account for the vast quantity of stuff that is licensed to the entire university community. The data you seek, plus much much more, is ready and waiting, and can be queried from your python code using whichever Postgres library you prefer :)
I hate to be that kind of person, but if you want to work with SEC filings “at scale”, you should use the bulk download link that the SEC provides. And if you want to get large lists of filings, you should use the master index files that the SEC provides. This EFTS API thing is really only good for checking what was filed on the current day, before the master index for the day is generated.
Other than that, it’s a very nice job. Oh, and if the GitHub account owner reads this, I would like to point out that GitHub clearly identifies which school you go to, and your blog (linked in the project) says that you created this project because it would have cost $35k to get access to the dataset. Well, that’s wrong. Over at the business school within your fine university, tell them you’d like a WRDS account for the vast quantity of stuff that is licensed to the entire university community. The data you seek, plus much much more, is ready and waiting, and can be queried from your python code using whichever Postgres library you prefer :)