The troubles of scraping a minecraft map.

i used to play on a minecraft server called CCNet, its a geopolitical earth server. Now heres the thing, the server has a towns system and in each town you have to pay an upkeep (heh taxes) everyday so that your town doesnt get deleted, therefore you have to deposit your money at your towns bank. now a nation has alot of towns and its reaally tedious to always check every town and see if it will die or not, That is why this exists (existed):

BEHOLD THE CONCEPT
Its a very simple concept, you give some inputs and you see all the towns banks and upkeep no manual hard work need.

Some additional information

The CCNet Map is created with Dynmap a minecraft plugin that dynamically creates a google maps like website for you

"note that if i have to make it much harder to scrape the dynmap, i will"
-Ineusia (CCNet Admin)

oh no here comes the realization

I used selenium and beautifulsoup to scrape the website, i couldve used the requests library but i opted not to for "a learning experience", now things were going smoothly i scraped the markers, the positions it was going well until i ran into a simple roadblock, i was actually clicking the nav elements instead of the actual town elements so i fixed it and "yay project finished!" haha. well then the next day i come back to see thatALL THE ELEMENTS COMPLETELY TELEPORTED TO RANDOM PLACESand not only that they had 0 ids and 0 unique classes to even identify each town now to fix this i was relying on scraping the position of the town instead of the XPaths of the element, This was working on my pc aswell but uh when i told people to test it none of it worked, it seemed _the position was unique and dependent on the browser and the screens resolution aswell* so even though it worked on my pc on my resolution on my browserit would not work for anyone else.soo i switched over to requests and what i found out isthe position now just does not exist??Wow, how hard is it to scrape this damn thing? from XPaths to Coordinates to "f*ck you". ig good job ccnet admins you prevented me from making a qol program :)

(Story is 5 months old)

The above is the detailed content of The troubles of scraping a minecraft map.. For more information, please follow other related articles on the PHP Chinese website!