Save cookies by simulating login, and then go to the backend page to continuously capture order data on a website.
Two questions now
The cookie on this website will expire in 10 minutes. After expiration, you need to log in again
Frequent querying of order data on this website is prohibited
Now we need to complete this function:
Starting from the simulated login, order data is continuously and automatically captured
When the cookie expires, a message will be automatically sent to the administrator’s WeChat to remind the administrator to log in again.
So, how to set the saved cookie expiration time during simulated login? And how to set up a data capture operation every fixed time?
Because this data needs to be processed, is there a way to complete these operations in local testing?
Save cookies by simulating login, and then go to the backend page to continuously capture order data on a website.
Two questions now
The cookie on this website will expire in 10 minutes. After expiration, you need to log in again
Frequent query of order data on this website is prohibited
Now we need to complete this function:
Starting from the simulated login, order data is continuously and automatically captured
When the cookie expires, a message will be automatically sent to the administrator’s WeChat to remind the administrator to log in again.
So, how to set the saved cookie expiration time during simulated login? And how to set up a data capture operation every fixed time?
Because this data needs to be processed, is there a way to complete these operations in local testing?
To execute crawling at fixed intervals, you can use cli mode to schedule tasks.
I don’t think it’s necessary to set the cookie expiration time. You can simulate login again if the interface is invalid.
1 Frequent querying of order data on this website is prohibited
curl supports proxy requests, and it detects frequent requests through IP
2 The cookie on this website will expire after 10 minutes, and you need to log in again after expiration (sessionid is stored in the cookie Mainly because the session id will expire, not the cookie)
According to your scenario, there are two types of cookie expiration:
----1 The session length exceeds the session length configured by the server (generally the default is 30 minutes)
----2 The user exits and the server actively destroys the session ID
Since you want to capture data, the account should be yours, so the possibility of 2 can basically be ruled out. As for 1, if you use curl proxy request, then there will be no problem with 1. If You can use a scheduled request to maintain the session ID (open another program to request every 15 minutes to maintain the session)