rest api - Scheduled parsing of XML feed into wp_insert_post()

admin管理员组

文章数量:1124680

I coded a custom theme that parses and imports into a CPT the content of some multi page XML feeds. This it the current workflow:

• daily action scheduled for 2am with action-scheduler to process all the feeds
• for each feed, parse it's URL to get all the pages in the feed, then enqueue (with action-scheduler) the import of the feed page
• in a single run, parse the XML feed (with SimpleXML), get each element from the feed, process it and ultimately insert the element data with wp_insert_post(), update_field() and wp_insert_term(); at the end, enqueue the import of the next feed page

To make sure these actions are running even without WP visitors:

• I set up a short script running via the cPanel cron and accessing a REST API endpoint - if the script receives 0, it stops, for any other code it sleeps for 30 seconds and then accesses the API endpoint again
• in WP, the function handling the API checks if there any actions need to be processed, in which case it returns a specific code, or 0 in case nothing is pending.

This worked well until I got to some 6k posts in WP. Then the whole process stopped in error when some pages exceeded 300 seconds of run time. Although I increased PHP's max_execution_time beyond this value, I found out that the 300 seconds hardcoded limit would be a feature (?!), and not a bug of action scheduler.

Combined with this, I also received the request to do the import more often than once per day. So now I intend to break my large single run into individual steps, one for each element / post.

For these individual steps, though, I am looking for something different than action scheduler, something that would allow me this procedure:

- Step N: run the step N processing, checks, inserts;
- Step N: call the step N+1 => if it answers OK, exit, else enqueue an action and exit
- Step N+1: answer OK
- Step N+1: run the step ...
...

Would this be possible in WP, and if YES, what tools should I look into. If not, what would be the best way to approach this?

本文标签：