I am using code that retrieves 100 elements from a website and inserts it into the database, if the element exists, it updates this entry. Every month it works as a cron job. It operates up to 50,000 products.
The script works well and the products are imported successfully. The problem now is loading the server and MySql. My server support stated that it takes up almost 50% of the resources and they need to disable this script.
I tried google to check the solution, and find that Load file is better than inserting and updating mysql_query. Therefore, I plan to write all the requests in a txt file on the server and, after import is complete, import it into db using the download file.
Is this a good method? or does it also take the same load on the server? Before continuing, I want to know an expert solution. Please help. Thanks
, . , - , , , . , MySQL, , , script, , cron ( , , , -). , : - , , .....? - , , .....? - - ? - ? ? - ? - Memcache?
, ? , cron , .
, , DB , . : , , WHERE, .
, .