Development XML in PHP: Big files cause problems?

Discussion in 'Software' started by jezmck, 17 Oct 2005.

  1. jezmck

    jezmck Minimodder

    Joined:
    25 Sep 2003
    Posts:
    4,456
    Likes Received:
    36
    Hi,
    I am trying to develop some fairly basic php which reads an xml file.
    (I'm using MiniXML, but any other PHP4 recommendations are welcome, xslt also required.)

    I have done that successfully for short(ened) XML files, but it seems that anything larger than 50kb causes the server to give up.
    What's worse is that it seems to simply quit without returning anything to the browser. (IE shows 404, Fx doesn't complain and doesn't even blank the page!)

    Should the file size matter? I've read stuff about people processing 20mb files, so problems with 50k seem ridiculous.

    Any ideas?
    TIA, j

    edit: it works with no problems on my own server, but I don't know if that's because it's PHP5, or perhaps because of some config option.
     
    Last edited: 17 Oct 2005
  2. acidfire

    acidfire What's a Dremel?

    Joined:
    11 May 2004
    Posts:
    259
    Likes Received:
    0
    It may be the php5, which host are you using? If your paying, you really should switch to a host who updates their server regularly. If you have a recent version of PEAR installed with php you can look at These

    they work very well, and quite quickly too.

    Maybe you post your xml parsing code, might be an issue there.
     
  3. jezmck

    jezmck Minimodder

    Joined:
    25 Sep 2003
    Posts:
    4,456
    Likes Received:
    36
    Unfortunately the host isn't my choice, and they were chosen for other reasons.
    If they'd install PHP5 there'd be no problem, but they say they won't do it.

    I don't think it's the parsing code since I'm literally copy and pasting the code from the MiniXML site, and the fact that it works for the smaller XML files.
     
  4. simon w

    simon w What's a Dremel?

    Joined:
    3 Nov 2003
    Posts:
    1,302
    Likes Received:
    0
    If it's not scaling well, then I guess it's a resource issue... Your not using recursion or all loads of nested loops are you?
     
  5. jezmck

    jezmck Minimodder

    Joined:
    25 Sep 2003
    Posts:
    4,456
    Likes Received:
    36
    Seems the likely thing, but the XML file really isn't that big, and I'm doing nothing more than loading the file and displaying a single element by its path.
    The host is no lightweight cheapy either, so it really shouldn't be that.
     
  6. simon w

    simon w What's a Dremel?

    Joined:
    3 Nov 2003
    Posts:
    1,302
    Likes Received:
    0
    * What's post_max_size and upload_max_filesize set at in PHP.ini? If these are too low, you may be able to override them with a .htaccess file

    * If you've got SSH access, check CPU and memory usage whilst uploading a file.

    * Try the code on another server?

    * Try doing something more simple (to discover *exactly* where the problem lies). Just load the file into an array and do a print_r() on it.

    * Failing that, post some sample code here for someone to look at.
     
  7. jezmck

    jezmck Minimodder

    Joined:
    25 Sep 2003
    Posts:
    4,456
    Likes Received:
    36
    those setting in the php.ini are unavailable/inaccessible to me :(
    but I'm working on a file already on the server anyway, so I guess those aren't relevant anyway.

    Code works fine on my own server, but as said, that might simply be because it's php5.

    trying to print_r() has the same (non) result. :(

    Code is ultra basic:
    PHP:
    <? require_once("minixml.inc.php"); 
    $xmlDoc = new MiniXMLDoc();
    $xmlDoc->fromFile("result.xml");
    $rootEl =& $xmlDoc->getRoot();
    $returnedElement =& $rootEl->getElementByPath('TimeSeriesCollection/Series/Item/Value');
    echo 
    $returnedElement->toString();
    ?>
    result.xml
     
  8. FuzzyOne

    FuzzyOne

    Joined:
    19 Sep 2002
    Posts:
    1,823
    Likes Received:
    32
    create a new php file with

    PHP:
    <?php phpinfo(); ?>
    in it and run that, you should be able to see what max upload is set to
     
  9. jezmck

    jezmck Minimodder

    Joined:
    25 Sep 2003
    Posts:
    4,456
    Likes Received:
    36
    "2M" which I assume means 2mb, but it's not an upload I'm working with.

    I am now trying to use Pear, but I can only get the first file to be seen (XML/Tree.php), but that complains that it can't see XML/Parser.php - even though I can see it myself and it's in the same dir. This is getting quite frustrating since it's the last piece of this puzzle.

    Thanks for all your help so far everyone btw.

    j
     
  10. RTT

    RTT #parp

    Joined:
    12 Mar 2001
    Posts:
    14,120
    Likes Received:
    74
    You can (usually - 99% of the time) issue php settings in a .htaccess file (assuming web and not shell php programming, and an apache server ;))

    Code:
    php_flag $php_global $value
    eg:

    Code:
    php_flag max_upload_filesize 1000M
    or any other var.

    Apache afaik usually over-rides the php upload size setting but i can't offhand remember the default. I shouldn't have thought you'd be dealing with signifcant file upload sizes either... but this may help someone else in the future :D

    edit: check those posts times!!

    --

    The only thing i can think of would be check your script isnt breaching the php maximum runtime level (30s i think default). And set error reportings to E_ALL to see the maximum possible error/warning output...
     
  11. jezmck

    jezmck Minimodder

    Joined:
    25 Sep 2003
    Posts:
    4,456
    Likes Received:
    36
    well it's just wierd. :grr:

    I've set all the suggested config's and some others, and none has made any difference. (I've also noticed that it doesn't seem to affect the values in phpinfo() so perhaps I'm not actually able to set them.)

    I have found that once the file size is below ~20k then it will work.
    Unfortunately the file size is out of my hands because it's coming from an external source.


    PEAR:
    I've found an error log reporting that:
    Code:
    [18-Oct-2005 22:11:11] PHP Warning:  Unknown(): Unable to load dynamic library './php_domxml.dll' - Cannot open &quot;./php_domxml.dll&quot; in Unknown on line 0
    "...in Unknown"?! :eyebrow:
    so for now I think I'll abandon PEAR - my host is being quite unhelpful saying that this is "web design issues" which they don't cover.
     
  12. FuzzyOne

    FuzzyOne

    Joined:
    19 Sep 2002
    Posts:
    1,823
    Likes Received:
    32
  13. jezmck

    jezmck Minimodder

    Joined:
    25 Sep 2003
    Posts:
    4,456
    Likes Received:
    36
    unfortunately my server is running FreeBSD, so I guess that can't be related.
     

Share This Page