0007410: Sending local URL by cron fails

Notes
(0016958) michiel (administrator) 16-08-06 17:13	yes, I think the same may apply to pulling RSS feeds from a local machine. I should be fixable with some server or network configuration though. It may not be fixable by doing something in phplist.

(0016959) saul11 (reporter) 16-08-06 17:20	Thanks for that incredibly fast response. Do you have an idea what kind of configuration that could be?

(0016963) michiel (administrator) 16-08-06 18:28	well, basically you'd have to go to the machine and try to load the site, eg "wget http://site.eg" [^] if that doesn't load, you'd need to put something in your hosts file (/etc/hosts) to fool the machine to think that the site is somewhere else. It'll be a bit of tweaking, but something should be possible.

(0017079) saul11 (reporter) 18-08-06 15:20	How exactly fetches PHPList the (local) URL when processing the queue as a cron job? Knowing this probably helps in the discussoin with my ISP...

(0017080) michiel (administrator) 18-08-06 15:32	it's getting it with PEAR::HTTP_Request no idea whether that's Socket coded, or just a "fopen". You would need to check the source of that module.

(0017135) saul11 (reporter) 19-08-06 15:15	I turned up that my ISP cannot change anything about it, so I'll have to try and change the source. Could you point me to the file and location I need to be in? Or could PHPList maybe fetch the page on creation of the message. That would avoid this problem Still another question, what's the difference (technically) between manually processing the queue and processing it as a cron? Why does a manually started process run fine?

(0017170) michiel (administrator) 21-08-06 13:57	well, that's one of those tricky things to depend on Pear Libraries. This is the only place in phplist that does, as I was too lazy to code URL fetching myself. Pear libraries are system wide, and therefore harder to change. You'd have to pull down your own version, hack it and tell phplist to use that one instead. All rather convoluted. Alternatively you could rewrite the "fetchURL" function in phplist to do it your own way, eg with fopen.

(0017515) saul11 (reporter) 27-08-06 15:17	I have changed the piece of code triggering the "fetchURL" function to first try to fetch the page locally, then only if that fails PHPList tries the PEAR::HTTP_Request method. Only thing is, when the page is fetched locally, it will not be cached. I'm not sure about how to do this so I went for the easy way. The code change is in file sendemaillib.php on line 150. I replaced: $remote_content = fetchUrl($url,$userdata); with: // first try to fetch locally global $domainName, $documentRoot; if (preg_replace("/^(http:\/\/)?([^\/]+)./i", "$2", $domainName) == preg_replace("/^(http:\/\/)?([^\/]+)./i", "$2", $url)) { $remote_content = file_get_contents($documentRoot . preg_replace("/^(http:\/\/)?[^\/]\/(.)/i", "$2", $url)); } // if nothing fetched, try remote if ($remote_content == false) { $remote_content = fetchUrl($url,$userdata); } For this to work, two new config settings are needed. So also in the config I made some changes. (You might find these setting odd and believe that this info is available to PHP anytime, but in my experience it isn't when running a script as a cron job.) I created a new section called 'custom settings' and I also moved the inclusion of the PEAR package to it, resulting in something like this: /* ========================================================================= Custom settings ========================================================================= */ # For sending webpages include the file Request.php of your PEAR::HTTP_Request library # e.g. require_once("/home/example.com/pear/HTTP/Request.php"); require_once("/home/example.com/pear/HTTP/Request.php"); # When you send local pages as a cron job, you might find all your mails getting failed # This might be due to server restriction of your ISP # If so, fill the first setting below with your domain name, e.g. "http://www.example.com" [^] # and the second setting with your server's document root, e.g. "/home/example.com/www/" # This will fetch the content locally instead of remote $domainName = "http://www.example.com"; [^] $documentRoot = "/home/example.com/www/";

(0017530) michiel (administrator) 28-08-06 17:38	Hmm, yes, but that trick will only work when the HTML page is a complete file on your server. If the page is dynamically constructed, eg from a database, then it won't work. Quite a tricky one to sort out.

(0017950) saul11 (reporter) 05-09-06 09:21	Is your proposal of "putting something in the hosts file (/etc/hosts) to fool the machine to think that the site is somewhere else" still an option? I'm not familiar with this, so if you could steer my an the right direction, that would be great. Okay it works now with the code change, but it's not that elegant...

(0017951) saul11 (reporter) 05-09-06 09:36	Or, like I asked before, is there a way PHPList can fetch the page on creation of the message?

(0050702) iSE (reporter) 03-08-09 09:48	Could you not just use: ob_start(); include('http://site.eg'); [^] $remote_content = ob_get_contents(); ob_end_clean(); ?