-
Apr 10th, 2016, 01:49 PM
#1
Thread Starter
Addicted Member
Scrape html with js and php?
Hello to all,
Last week I started work on a parser in php to take the data from soccerway.com, the problem is that php as well as being a server-side language (thus some content that you upload later I can not pick them up), it is also very bare to parser level. The few that there are are full of bugs and slow, and in my case with simple_html_dom you eat a lot of memory on the server because it is totally wrong.
So I'm going to write a parser in Javascript for the website at the top. I'm going to base myself on the MVC model, the idea would be:
1. A cron job php launches a JavaScript file that includes all classes with methods for each section which scrapare Site
2. Any data collected from js is sent to php file that will take care of loading data on my database
3. When an update is done to the log file
What I want to ask you is if you can recommend a good parser in js (or a framework) that allows me to do this without much effort and in the manner as clean as possible, I do not know if you know CodeIgniter or Slim, these two frameworks provide methods for queries, so there is no need to write the query every time, but that's another story.
I hope everything is clear
-
Apr 10th, 2016, 05:04 PM
#2
Re: Scrape html with js and php?
Just download the jQuery library and you will be traversing the DOM in JavaScript like there was no tomorrow!
I've never used CodeIgniter or Slim - they might be more lightweight.
-
Apr 10th, 2016, 06:13 PM
#3
Re: Scrape html with js and php?
CodeIgniter is a PHP-based MVC framework.
Honestly... I'm struggling a bit on how this is supposed to work... For example the CRON job... you're expecting it to run some JS ... where? on the server? Is that even possible?
-tg
-
Apr 10th, 2016, 06:35 PM
#4
Re: Scrape html with js and php?
I'm with TG here - how are you going to get the external HTML from another site to be readable by the JavaScript running on your page?
-
Apr 10th, 2016, 09:23 PM
#5
Re: Scrape html with js and php?
Perhaps you might want to look at Node.js then? There's a few node modules available for doing web scraping.
-
Apr 11th, 2016, 05:57 AM
#6
Thread Starter
Addicted Member
Re: Scrape html with js and php?
For run javascript code from php is at least a php extension that embed (or wrap arround) a Javascript engine, and as a consequence, allows one to execute Javascript on the server, from PHP.
The extension I'm thinking about is the spidermonkey
I'm going to install it on the server, enabling it and will allow me to execute Javascript code, on the server, from PHP.
Of course, like any other PHP extension, I need to be admin of the server, in order to install it.
My goal as I said is use curl with php, pass the html grabbed to js and deserialize it. Some idea?
-
Apr 11th, 2016, 06:19 PM
#7
Re: Scrape html with js and php?
If you want javascript on the server, just go straight to Node.js?
Posting Permissions
- You may not post new threads
- You may not post replies
- You may not post attachments
- You may not edit your posts
-
Forum Rules
|
Click Here to Expand Forum to Full Width
|