Eve Market is built in Laravel 5.2 and uses the Eve API’s from Public crest to fetch the data. There are some tasks that run from every day to every month to fetch data incase things change.
Right now we fetch the Market Prices, which is over 13 thousand items and thats fetched at 1 AM EST time.
We also fetch regions every month, we also fetch the Region Buy and Sell order on demand, that is when you click on a region to see who's buying or selling said item we grab that on demand.
We have the ability to search every region to see who's buying or selling an item, note: this can take up to two minutes.
Aside from that we also have market groups, which are groups of items that make finding things a little easier.
What was working with the eve api like?
A nightmare, to be honest. It’s not well documented, its just a bunch of links that take you to jon responses that have links that take you to there relationships, so architecting the whole system was a nightmare.
Right now we have the following tables:
- Eve Market Types: This contains all the items, if it doesn't exist, we create when we fetch information from market groups api end point.
- Eve Market Groups: Contains the Market group Name, item id, maps to Eve Market Types and the slug name of the group. We only fetch this once per application life cycle. In the future we might fetch this more then once, but for not its once, it requires a a task to be run and left up while we are not connected to a server instance, we then make multiple calls to the api end point of each group to get the items, this is done via jobs and can take up to 5-30 minutes.
- Eve Regions: Just the region names, used for searching for buy and sell orders.
- Eve Region Orders: Stores the item for the region under a buy or sell section and allows us to collect data. Data here can duplicate as the data is fetched ONLY ON DEMAND.
These are the core tables that store data. Right now I believe that Eve market types has something like 36k+ pieces of unique data.
I would like to change the name of the Eve Market Types to Eve Items, as thats all it is, a lit of items and there associated details.
Whats fetched on demand?
When you click on an item to get more info, we fetch that if it’s not already in the database, all tags and html is stripped from the description text.
When you click on “Who’s Buying” or “Who’s Selling” or even “Search all regions” we get that and store it, regardless of duplicates, from the api end point.
This is the slow part of the whole service. Telling us that you want us to search all regions for the item to see who's buying or selling requires us to loop over all regions looking at the market buy or sell orders looking for that specific item, construct a data object, store it and then show you the information you requested.
I am looking into ways to make this faster, how ever at this time fetch on demand is the fastest way, there is some additional data from the region information we can store to make this process a smudge faster.
Wheres search? or filtering?
Right now this whole system is a proof of concept. That is I wanted the basics out there. In the future I am looking at adding additional features including community requests and so on.
I am also looking at open sourcing a peace of the system, the part that fetches the data from public crest.
Right now its about stabilizing whats here, cleaning up the back end a bit, testing functionality and seeing what the community thinks so far.