Proof

• New functionality
  • In TProof::Load, add the possibility to define a list of files to be sent because needed by the main macro file. The list is comma-separated and the first file is considered as the main one. For example
    
            proof->Load("<macropath>/mymacro.C+,<thispath>/thisheader.h,<thatpath>/thatheader.h")
    
    will make sure that the files 'thisheader.h' and 'thatheader.h', needed by 'mymacro.C' are available in the sandbox on the worker machines. Note that 'thisheader.h' and 'thatheader.h' will be available remotely in the sandbox, as 'mymacro.C'; so they should be included directly by 'mymacro.C', e.g. '#include "thisheader.h"' .
  • Import the dataset stager daemon 'afdsmgrd' into ROOT; this is used to manage data staging based on the dataset information (see http://code.google.com/p/afdsmgrd/ for more info). The daemon is located under $ROOTSYS/proof/afdsmgrd .
  • New PROOF bench suite, a framework to run CPU and IO benchmarks with default selectors/data or with user-provided ones. The code is located under proof/proofbench. See http://root.cern.ch/drupal/content/new-benchmark-framework-tproofbench .
  • Add the possibility to access the files on the workers via the same port used by PROOF. This is useful for cases when it is not possible to start a file server daemon on a different port (because, for eample, of a firewall or just inconvenience) and workers do not share a file system. Internally this works by forking a 'rootd' after identifying a file request and trasferring the connection to it. The client side is a TNetFile and it is triggered by the protocol "rootd://" (the just implemented etc/plugins/TFile/P120_TNetFile.C includes this protocol).
    See also http://root.cern.ch/drupal/content/configuration-reference-guide#rootd .
  • Add support for log file truncation. Truncation is disabled by default. Enabling is controlled by the rootrc variable
    
               ProofServ.LogFileMaxSize   {<bytes>|<kilobytes>K|<megabytes>M|<gigabytes>G}
    
     indicating the max number of bytes. The number can be followed by a {K,M,G} to indicate KBytes, MBytes or GBytes, respectively.
  • Add new derivation of TList (TProofOutputList) to be used on the PROOF client to filter out PROOF internal objects when displaying or printing the list. By default objects was names start with 'PROOF_' are not shown. The presence of a non empty missing file list is notified.
  • In the PROOF monitoring to: send additional information about memory usage during the query, the name and size (# of files) of the dataset processed (if any); add possibility to send the information to multiple monitoring collectors. See documentation updated at http://root.cern.ch/drupal/content/enabling-query-monitoring .
  • Add support for block activation/deactivation of workers.
  • Add possibility to start the proofserv with 'system()' instead of 'fork()' as done in PROOF-Lite. A new switch 'usefrk' has been added to 'xpd.proofservmgr' to control that. Default is still fork().
• Improvements
  • In TProof::ClearPackages, use the manager to execute the command on all known worker machines. Improves the consistency when re-istalling packages.
    
  • In TProof::GetDataSets, add support for option ':lite:'; this allows to fill the map with only the summary information about the datasets (the header of TFileCollections), significantly increasing the speed and the memory footprint when the number of datasets is very large.
  • Accept '.' in user names.
  • Add switch to control caching of the files read on MacOsX. A call to fcntl(fd, F_NOCACHE, 1) is done after opening the file.
  • Add export of the envs ROOTPROOFCLIENT and ROOTPROOFLITE when appropriate. These allow to steer building and/or enabling of PAR files in PROOF-INF/BUILD.sh and/or PROOF-INF/SETUP.C, improving transparency between normal ROOT and PROOF. The example PAR 'tutorials/proof/event.par' has been modified to check the two variables.
  • Fix a few issues in SQL PROOF monitoring: in TSQLMonitoringWriter::SendParameters, drop ''' around field names in the INSERT string; also use TString::Format(...) instead of Form(...) where relevant.  In TPerfStats: call 'proofgroup' instead of 'group' the field with the PROOF group (interference with the 'group' keyword in SQL); add new field 'querytag' VARCHAR(64) with the unique query tag; in WriteQueryLog fill also the field 'totevents'; in PacketEvent, add switch to control whether to send te information to the monitoring system on per packet level (may be too much for SQL). The switch is called fMonitorPerPacket and it is globally controlled by the rootrc variable 'Proof.MonitorPerPacket' and at session level with the parameter PROOF_MonitorPerPacket .
  • Improve treatment of the case when temporary files are asked to be created on a shared file system not containing the sandboxes. This case, which seems to be a rather common one, should be now fully supported.
  • Correctly honour selector abort status settings TSelector::kAbortProcess and TSelector::kAbortFile.
  • Improve reporting of the non-processed {files, events} in the final 'MissingFiles' list. 
  • Improved algorithm for TPacketizerUnit to fix issue with non homogeneous machines.
  • Improve the way the information about log files is saved in case of failures. The log paths for these failing now should be now correctly saved and accessible via TProofLog.
  • Improve merging of histograms. Just use TH1::Add whne the axis are equal; much faster than TH1::Merge.
• Fixes
  • In TDataSetManagerFile::NotifyUpdate fix handling of the case when the global list file does not exist yet (new dataset directory). Fixes error messages during editing dataset operations.
  • Fix issue with machine names consistency when working on a local machine ('localhost' or 'localhost.localdomain' are mapped to gSystem->HostName()); solves possible matching problems in the packetizer.
  • In TProofServ, fill the "grand total" message with more blanks, so that no remnants of the previous message are visible on the screen.
  • In the autoconf/bonjour interface, fix issue preventing the correct port (the protocol one, which may be different from the application default) being used when registering the service.
    
  • In TProofPlayer::AddQueryResult, fix a bug affecting the order in whihc query results are registered when the start time is within 1 second.
  • Fix worker name in TSlaveLite.
  • Fix problem with enabling packages with option 'notOnClient' in PROOF-Lite .
  • Make sure the log file is flushed at the end of startup to avoid spurious log messages on next commands .
  • In CreateSession(), fix an issue with the validity check for existing sessions .
  • In TProofLite: fix problem with passing the 'varexp' and 'selection' strings for processing, preventing correct usage of the operators '|' and '||' in TTreeFormula.
  • In the TProofOutputFile constructor, remove the 'localroot' prefix only if present in the path. Fixes possible truncation problems occuring when the paths are not under the localroot scope.
  • In TXSocket and TXSlave: fix problem with the way collection over a socket just marked as 'bad' was interrupted; the interrupt was de facto ineffective, so that collection stayed always until the timeout expired (default: 5 minutes). Should solve some of the cases were slow response was experienced.
  • Fix a problem with log path transmission when the node dies early or not even starts. The log path was empty and wrong was filled in when retrieving the log buffers, disorienting debugging.
  • Fix a bug checking the first event which rendered ineffective the request for processing a subset of events in a given dataset or chain.
  • In pq2-ana-dist, fix problem with the labels of the distribution histo occuring when machines are represented by IPs instead of names.
  • Add missing calls to closedir() and TSystem::FreeDirectory, cuasing a large number of filedescriptors remaining opened after xproofd initialization.
  • Fix a problem with the final update of the progress information affecting occasionally cases with skipped events.
  • Fix merging of TproofOutputFile when using submergers (the intermediate files were not correctly handled).
  • Fix the way TChain weights are transmitted to TProofDraw in DrawSelect operations.
• AoB
  • The class TFileMerger has been moved to 'io/io'