getURL.RdThese functions download one or more URIs (a.k.a. URLs). It uses libcurl under the hood to perform the request and retrieve the response. There are a myriad of options that can be specified using the ... mechanism to control the creation and submission of the request and the processing of the response.
getURLContent has been added as a high-level function
like getURL and getBinaryURL but which
determines the type of the content being downloaded
by looking at the resulting HTTP header's Content-Type
field. It uses this to determine whether the bytes
are binary or "text".
The request supports any of the facilities within the
version of libcurl that was installed.
One can examine these via curlVersion.
getURLContent doesn't perform asynchronous or multiple
concurrent requests at present.
getURL(url, ..., .opts = list(),
write = basicTextGatherer(.mapUnicode = .mapUnicode),
curl = getCurlHandle(), async = length(url) > 1,
.encoding = integer(), .mapUnicode = TRUE)
getURI(url, ..., .opts = list(),
write = basicTextGatherer(.mapUnicode = .mapUnicode),
curl = getCurlHandle(), async = length(url) > 1,
.encoding = integer(), .mapUnicode = TRUE)
getURLContent(url, ..., curl = getCurlHandle(.opts = .opts), .encoding = NA,
binary = NA, .opts = list(...),
header = dynCurlReader(curl, binary = binary,
baseURL = url, isHTTP = isHTTP,
encoding = .encoding),
isHTTP = length(grep('^[[:space:]]*http', url)) > 0)a string giving the URI
named values that are interpreted as CURL options governing the HTTP request.
a named list or CURLOptions object identifying the
curl options for the handle. This is merged with the values of ...
to create the actual options for the curl handle in the request.
if explicitly supplied, this is a function that is called with a single argument each time the the HTTP response handler has gathered sufficient text. The argument to the function is a single string. The default argument provides both a function for cumulating this text and is then used to retrieve it as the return value for this function.
the previously initialized CURL context/handle which can be used for multiple requests.
a logical value that determines whether the download
request should be done via asynchronous, concurrent downloading or a serial
download. This really only arises when we are trying to download
multiple URIs in a single call. There are trade-offs between
concurrent and serial downloads, essentially trading CPU cycles
for shorter elapsed times. Concurrent downloads reduce the overall
time waiting for getURI/getURL to return.
an integer or a string that explicitly identifies the
encoding of the content that is returned by the HTTP server in its
response to our query. The possible strings are
‘UTF-8’ or ‘ISO-8859-1’
and the integers should be specified symbolically
as CE_UTF8 and CE_LATIN1.
Note that, by default, the package attempts to process the header of
the HTTP response to determine the encoding. This argument is used
when such information is erroneous and the caller knows the correct
encoding.
The default value leaves the decision to this default mechanism.
This does however currently involve processing each line/chunk
of the header (with a call to an R function). As a result,
if one knows the encoding for the resulting response,
specifying this avoids this slight overhead which is probably
quite small relative to network latency and speed.
a logical value that controls whether the resulting text is processed to map components of the form \uxxxx to their appropriate Unicode representation.
a logical value indicating whether the caller knows
whether the resulting content is binary (TRUE) or not
(FALSE) or unknown (NA).
this is made available as a parameter of the function
to allow callers to construct different readers for processing
the header and body of the (HTTP) response.
Callers specifying this will typically only adjust the
call to dynCurlReader, e.g. to specify a
function for its value parameter to
control how the body is post-processed.
The caller can specify a value of TRUE or FALSE
for this parameter. TRUE means that the header will be
returned along with the body; FALSE corresponds to the
default and only the body will be returned.
When returning the header, it is first parsed via
parseHTTPHeader,
unless the value of header is of class AsIs.
So to get the raw header, pass the argument as header = I(TRUE).
a logical value that indicates whether the request an HTTP request. This is used when determining how to process the response.
If no value is supplied for write,
the result is the text that is the HTTP response.
(HTTP header information is included if the header option for CURL is
set to TRUE and no handler for headerfunction is supplied in
the CURL options.)
Alternatively, if a value is supplied for the write parameter,
this is returned. This allows the caller to create a handler within
the call and get it back. This avoids having to explicitly create
and assign it and then call getURL and then access the result.
Instead, the 3 steps can be inlined in a single call.
Curl homepage https://curl.se/
omegahatExists = url.exists("https://www.omegahat.net")
# Regular HTTP
if(omegahatExists && requireNamespace("XML", quietly = TRUE)) withAutoprint({
txt = getURL("https://www.omegahat.net/RCurl/")
## Then we could parse the result.
XML::htmlTreeParse(txt, asText = TRUE)
})
#> > txt = getURL("https://www.omegahat.net/RCurl/")
#> > XML::htmlTreeParse(txt, asText = TRUE)
#> $file
#> [1] "<buffer>"
#>
#> $version
#> [1] ""
#>
#> $children
#> $children$html
#> <html>
#> <head>
#> <link rel="stylesheet" href="http://www.omegahat.org/OmegaTech.css"/>
#> <title>RCurl</title>
#> </head>
#> <body>
#> <h1>The RCurl Package</h1>
#> <p align="right">
#> <a href="RCurl_1.96-0.tar.gz">RCurl_1.96-0.tar.gz</a>
#> (20 June 2014)
#> </p>
#> <p align="right">
#> <a href="philosophy.html">Manual</a>
#> </p>
#> The RCurl package is an R-interface to the
#> <a href="http://curl.haxx.se">libcurl</a>
#> library that provides HTTP facilities. This
#> allows us to download files from Web servers, post forms, use HTTPS
#> (the secure HTTP), use persistent connections, upload files, use
#> binary content, handle redirects, password authentication, etc.
#> <p>The primary top-level entry points are</p>
#> <ul>
#> <li>
#> <a href="installed/RCurl/html/getURL.html">getURL()</a>
#> </li>
#> <li>
#> <a href="installed/RCurl/html/getURL.html">getURLContent()</a>
#> </li>
#> <li>
#> <a href="installed/RCurl/html/postForm.html">getForm()</a>
#> </li>
#> <li>
#> <a href="installed/RCurl/html/postForm.html">postForm()</a>
#> </li>
#> </ul>
#> However, access to the C-level routines is also available
#> via the R code, and one can specify options to all of the
#> libcurl operations to control how they are performed.
#> Documentation about the options and commands
#> can be found at the
#> <a href="http://curl.haxx.se">libcurl web site</a>
#> <p>
#> R functions can be specified to collect text from both the
#> response and its headers. This can be used to customize the processing
#> of the requests and feed the results to higher-level processing
#> (e.g. HTML parsing via the htmlTreeParse function in the
#> <a href="http://www.omegahat.org/RSXML">XML package</a>
#> ).
#> </p>
#> <p>
#> This package will be used to implement the low-level communication
#> in the
#> <a href="http://www.omegahat.org/SSOAP">SSOAP</a>
#> package
#> and other high-level packages that utilize HTTP to exchange
#> requests and data.
#> </p>
#> <h2>Documentation</h2>
#> <dl>
#> <dt>
#> <li>
#> <a href="RCurlJSS.pdf">Paper</a>
#> outlining the package with some advanced examples.
#> <dd/>
#> <dt>
#> <li>
#> <a href="philosophy.html">Guide</a>
#> <dd/>
#> <dt>
#> <li>
#> <a href="Changes.html">Changes across releases</a>
#> <dd/>
#> <dt>
#> <li>
#> Examples of using asynchronous, multiple concurrent requests.
#> <dd>
#> <ul>
#> <li>
#> <a href="concurrent.html">Concurrent downloads</a>
#> </li>
#> <li>
#> <a href="nestedHTML.html">Nested HTML requests</a>
#> </li>
#> <li>
#> <a href="xmlParse.html">XML parsing with nested requests</a>
#> </li>
#> </ul>
#> </dd>
#> <dt>
#> <li>
#> <a href="FAQ.html">FAQ</a>
#> <dd/>
#> </li>
#> </dt>
#> </li>
#> </dt>
#> </li>
#> </dt>
#> </li>
#> </dt>
#> </li>
#> </dt>
#> </dl>
#> <h2>Other Approaches</h2>
#> <dl>
#> <dt>
#> <a href="http://cran.r-project.org/src/contrib/Descriptions/httpRequest.html">httpRequest</a>
#> </dt>
#> <dd>The httpRequest is a package on CRAN that implements a small
#> part of HTTP directly in R using sockets.</dd>
#> <dt>httpClient</dt>
#> <dd>
#> I have developed the
#> <b>httpClient</b>
#> package using
#> R code and connections that supports additional
#> aspects of R and HTTP, such as cookies, character escaping, and also
#> SSL for HTTPS. I haven't released the code (favoring the
#> approach of building on existing C code) but can make it available if anyone
#> is interested.
#> </dd>
#> </dl>
#> While having code in R makes it easier to understand, explore and
#> modify, it is probably better to use existing specialized libraries
#> like libcurl rather than doing this ourself. We gain speed and a
#> large development community that cares about getting things right and
#> testing them.
#> We will explore the use of
#> <a href="http://www.w3.org/Library">libwww</a>
#> <h2>Issues</h2>
#> Using the opaque data structures of the libcurl infrastructure
#> means that we cannot easily access the file descriptors used
#> in the communication. This makes it somewhat more difficult
#> to integrate these streams into an R even loop
#> (e.g.
#> <a href="http://www.omegahat.org/REventLoop">REventLoop</a>
#> ).
#> We can potentially turn them into regular connections
#> (if the internal API is made "public").
#> <h2>License</h2>
#> This is distributed under the
#> <a href="http://www.omegahat.org/BSDLicense.html">BSD license</a>
#> in the same spirit as libcurl itself.
#> <hr/>
#> <address>
#> <a href="http://www.stat.ucdavis.edu/~duncan">Duncan Temple Lang</a>
#> <a href="mailto:duncan@wald.ucdavis.edu"><duncan@wald.ucdavis.edu></a>
#> </address>
#> <!--hhmts start-->
#> Last modified: Mon May 25 11:35:38 PDT 2009
#> <!--hhmts end-->
#> </body>
#> </html>
#>
#>
#> attr(,"class")
#> [1] "XMLDocumentContent"
# HTTPS. First check to see that we have support compiled into
# libcurl for ssl.
if(interactive() && ("ssl" %in% names(curlVersion()$features))
&& url.exists("https://sourceforge.net/")) {
txt = tryCatch(getURL("https://sourceforge.net/"),
error = function(e) {
getURL("https://sourceforge.net/",
ssl.verifypeer = FALSE)
})
}
# Create a CURL handle that we will reuse.
if(interactive() && omegahatExists) {
curl = getCurlHandle()
pages = list()
for(u in c("https://www.omegahat.net/RCurl/index.html",
"https://www.omegahat.net/RGtk/index.html")) {
pages[[u]] = getURL(u, curl = curl)
}
}
# Set additional fields in the header of the HTTP request.
# verbose option allows us to see that they were included.
if(omegahatExists)
getURL("https://www.omegahat.net", httpheader = c(Accept = "text/html",
MyField = "Duncan"),
verbose = TRUE)
#> [1] "<!DOCTYPE HTML PUBLIC \"-//IETF//DTD HTML//EN\">\n<html> <head>\n<title>The Omega Project for Statistical Computing</title>\n<link rel=stylesheet href=\"My.css\" >\n<LINK rel=\"alternate\" TYPE=\"application/rss+xml\"\n TITLE=\"RSS\" HREF=\"/omega.rss\">\n</head>\n\n<body>\n<h1><a name=toc1>The Omega Project for Statistical Computing</a></h1>\n<!--\n<p align=right><a href=\"omega.rss\"><img src=\"rssfeed.gif\"/><font size=-1>RSS of recent topics</font></a>\n-->\n\n<p align=right><a href=\"http://omegahat.wordpress.com/\">Blog</a></p>\n\n<p align=right><a href=\"contents.html\"><font size=-1>Table of Contents</font></a>\n<!--<p align=right><a href=\"Wiki\"><img src=\"Images/new.jpg\"><font size=-1>Wiki for discussing\n Omegahat packages and ideas.</font></a>-->\n<!--\n<p align=right><a href=\"RecentActivities.html\"><font size=-1>Recent Activities</font></a>\n-->\n<p align=right><a href=\"cranRepository.html\">CRAN-style repository for Omegahat packages</a>\n\n<p align=right><a href=\"Papers/index.html\">Papers on General Topics</a>\n\n<!--<p align=right><font size=-1\">rsync access to CVS and Web trees: <code>rsync www.omegahat.net::</code></font>-->\n\n\n<h2>Recently Updated packages</h2>\n<div style=\"padding-left: 50pt; padding-right: 50pt;\">\n<div style=\"background-color: lightgray;\">\n<p>\n These can (typically) be download from the package-specific site\nwhich is <i>http://www.omegahat.net/<package name></i>,\ne.g.\n <b>http://www.omegahat.net/RGraphicsDevice</b>\n\n<p>\nAlternatively, you can download any package (and its previous versions) from the R repository on this site\n<center>\n <code>http://www.omegahat.net/R</code>\n</center>\nand specifically\n<center>\n <code><a href=\"http://www.omegahat.net/R/src/contrib\">http://www.omegahat.net/R/src/contrib</a></code>\n</center>\nOne can use this repository from within R via\n<pre class=\"scode\">\n install.packages(packageName, repos = \"http://www.omegahat.net/R\")\n</pre>\n\nFor some packages, there are binaries for Windows.\nFor most others, you will need to specify to download the source package:\n<pre class=\"scode\">\n install.packages(packageName, repos = \"http://www.omegahat.net/R\"<b>, type = \"source\"</b>)\n</pre>\n</p>\n</div>\n</div>\n\n <center>\n <table border=\"1\">\n <title>Omegahat Packages for R</title>\n\n <tr align='left'><th><img src=\"Images/new.jpg\"><a\n href=\"RFirefox\">RFirefox</a></th><th>\n This embeds R within the Firefox Web browser and allows \n JavaScript code to call R functions and evaluate R expressions.\n It also allows R functions and code to access JavaScript objects\n and methods and so be used as event handlers and to generate dynamic content within\n a Web page. (Gabe Becker and Duncan Temple Lang)\n </th>\n </tr>\n\n <tr align='left'><th><img src=\"Images/new.jpg\"><a\n href=\"RKML\">RKML</a></th><th>(Version 0.1-0) updates to RKML\n and <a href=\"RKML/JSM2011/README.html\">annotated examples</a> presented at the JSM 2011 in the\n session <a href=\"http://www.amstat.org/meetings/jsm/2011/onlineprogram/ActivityDetails.cfm?SessionID=206267\">Statistical Graphics in Climate Research</a>.\n </th>\n </tr>\n\n <tr align='left'><th><img src=\"Images/new.jpg\"><a\n href=\"RGoogleStorage\">RGoogleStorage</a></th><th>(Version 0.1-0) a package to interface\nGoogle's storage API from within R, to upload and download data.\n </th>\n </tr>\n\n <tr align='left'><th><img src=\"Images/new.jpg\"><a\n href=\"REuPathDB\">REuPathDB</a></th><th>(Version 1.0-0) a package to interface\nthe EuPathDB (Eukaryotic Pathogen Database Resources) Web Services\nvia their REST API. This is machine generated code derived from\nthe WADL files describing the Web service methods. \n </th>\n </tr>\n\n <tr align='left'><th><img src=\"Images/new.jpg\"><a\n href=\"WADL\">WADL</a></th><th>(Version 0.2-0) a package that can\nprogrammatically generate R code to interface REST Web Services\nthat are described by a WADL (Web Application Description Language)\ndocument, e.g. the EuPathDB.\n </th>\n </tr>\n\n <tr align=left><th><img src=\"Images/new.jpg\"><a href=\"RExcelXML\">RExcelXML</a>\n\t</th><th>Enhancements for reading XLSX files. See also the xlsx package by Adrian Dragulescu.</th></tr>\n\n <tr align='left'><th><img src=\"Images/new.jpg\"><a\n href=\"RGoogleDocs\">RGoogleDocs</a></th><th>update for RGoogleDocs\nto enhance uploadDoc() and deleteDoc() and subsetting worksheet references, among other things.\n </th>\n </tr>\n\n\n <tr align='left'><th><img src=\"Images/new.jpg\"><a\n href=\"REuPathDB\">REuPathDB</a></th><th>a package to interface\nthe EuPathDB (Eukaryotic Pathogen Database Resources) Web Services\nvia their REST API. This is machine generated code derived from\nthe WADL files describing the Web service methods. \n </th>\n </tr>\n\n <tr align='left'><th><img src=\"Images/new.jpg\"><a\n href=\"WADL\">WADL</a></th><th>a package that can\nprogrammatically generate R code to interface REST Web Services\nthat are described by a WADL (Web Application Description Language)\ndocument, e.g. the EuPathDB.\n </th>\n </tr>\n\n <tr align='left'><th><img src=\"Images/new.jpg\"><a\n href=\"SpiderMonkey\">SpiderMonkey</a></th><th>a package that maps allows R programmers to call\nJavaScript functions and methods from R and to evaluate JavaScript scripts.\nThis is useful for running JavaScript code that might be of use in R,\nbut also for manipulating Web documents displayed in a Web browser\nwhen R is embedded in the browser as an extension. This is possible\nwith an updated version of the RFirefox package to be released very soon.\n </th></tr> \n\n <tr align='left'><th><img src=\"Images/new.jpg\"><a\n href=\"ROpenOffice\">ROpenOffice</a></th><th>update for\nreading OpenOffice spreadsheets directly in R via the XML\ncontent in the .ods files.\n </th></tr> \n\n\n\n <tr align='left'><th><img src=\"Images/new.jpg\"><a\n href=\"GeoIP\">GeoIP</a></th><th>a package that maps IP addressess and \n host names to geographic locations - latitude, longitude, region, city, zip code, ....\n This package is built via RGCCTUFFI to provide dynamic access to existing native code.\n In addition to the basic functions provided by the C library, the package provides a function\n for finding the nearest CRAN mirror.</th></tr> \n\n <tr align='left'><th><img src=\"Images/new.jpg\"><a\n href=\"RGCCTUFFI\">RGCCTUFFI</a></th><th>a package that programmatically\ngenerates Rffi-based dynamic interfaces to native C routines and data structures\nwithout very much human intervention. (You have to call the main function in the package!)</th></tr> \n\n <tr align='left'><th><img src=\"Images/new.jpg\"><a\n href=\"RAutoGenRunTime\">RAutoGenRunTime</a></th><th>run-time support for\nprogrammatically generated code via RGCCTranslationUnit and RGCCTUFFI.</th></tr> \n\n <tr align='left'><th><img src=\"Images/new.jpg\"><a\n href=\"Rffi\">Rffi</a></th><th>an interface to libffi for invoking\n arbitrary compiled routines from R without the need for writing and compiling\n wrapper code. Instead, one can describe the routine's input and output types in R\n and Rffi does the marshalling to invoke the routine and return the value.</th></tr> \n\n\n <tr align='left'><th><img src=\"Images/new.jpg\"><a\n href=\"RGCCTranslationUnit\">RGCCTranslationUnit</a></th><th>functionality to read C/C++ code and to generate\nbindings to those routines, classes and data structures.</th></tr> \n\n\n <tr align='left'><th><a\n href=\"XMLRPC\">XMLRPC</a></th><th>Bug fix release thanks to Paul Shannon.</th></tr> \n\n <tr align='left'><th><a\n href=\"RCurl\">RCurl</a></th><th>Bug fix for ftpUpload(), found by Spencer Graves.</th></tr> \n\n <tr align='left'><th><a\n href=\"RKML/GoogleEarth\">Google Earth plugin and R</a></th><th>Examples for embedding\n Google Earth in a Web page and using an R plot (in SVG) to\n interact with the Google Earth plugin. </th></tr> \n \n <tr align='left'><th><a\n href=\"RWordpress\">RWordpress</a></th><th>Facilities for\n interacting with a blog site.</th></tr> \n \n <tr align='left'><th><a href=\"XMLRPC\">XMLRPC</a></th><th>Update to\n the XMLRPC package for remote procedure calls via HTTP and XML.</th></tr> \n\n \n <tr align='left'><th><a href=\"RKMLDevice\">RKMLDevice</a></th><th>R\n graphics device (pure R code) for drawing R plots in KML for\n display on Google Earth.</th></tr> \n\n <tr align='left'><th><a href=\"RKML\">RKML</a></th><th>Updated</th></tr>\n \n <tr align='left'><th><a href=\"RHTMLForms\">RHTMLForms</a></th><th>Updated</th></tr>\n <tr align=\"left\"><th><a\n href=\"RFreetype\">RFreetype</a></th><th>A package to read font\n files,\n\te.g. True Type Font files (.ttf files),\n and to retrieve font metric information. This can be used\n by graphics devices (written in R with <a href=\"RGraphicsDevice\">RGraphicsDevice</a>)\n to do computations on strings to determine their sizes.</th> \n \n <tr align=\"left\"><th><a\n href=\"RGoogleTrends\">RGoogleTrends</a></th><th>Interface to query\n and download Google trends data.</th> \n \n <tr align=\"left\"><th><a href=\"RTidyHTML\">RTidyHTML</a></th><th>Interface to libtidy for\n cleaning up HTML documents to be well-formed</th> \n <tr align=\"left\"><th><a href=\"RNYTimes\">RNYTimes</a></th><th>Interface to several New York Times Web services</th>\n \n <tr align=left><th>\n\tREST-related packages\n\t</th>\n <th>\n The following are examples of implementing interfaces\n to REST APIs as an R client:\n <ul>\n\t <li> <a href=\"RGoogleDocs\">RGoogleDocs</a>\n\t <li> <a href=\"Zillow\">Zillow</a>\n\t <li> <a href=\"RAmazonS3\">RAmazonS3</a>\n\t <li> <a href=\"RAmazonDBREST\">RAmazonDBREST</a>\t \n\t <li> <a href=\"Rflickr\">Rflickr</a>\n\t <li> <a href=\"RNYTimes\">RNYTimes</a>\t \n\n\t <li> <a\n href=\"http://cran.r-project.org/web/packages/twitteR/index.html\">twitteR\n by Jeff Gentry</a>\n\t <li> <a\n href=\"http://cran.r-project.org/web/packages/RLastFM/index.html\">\n\t RLastFM by Greg Hirson</a>\n </ul>\n\n Related infrastructure packages include\n <ul>\n\t <li> <a href=\"RCurl\">RCurl</a>\n\t <li> <a href=\"XML\">XML</a>\n\t <li> <a href=\"RJSONIO\">RJSONIO</a>\n\t <li> <a href=\"http://cran.r-project.org/web/packages/rjson/index.html\">rjson</a>\t \n </ul>\n \n </th>\n </tr>\n \n <tr align=left><th><img src=\"Images/new.jpg\"><a href=\"RAmazonDBREST\">RAmazonDBREST</a>\n\t</th><th>A REST-based interface to Amazon's SimpleDB service\n\tfor hosting databases and facilitating creating databases,\n\tadding and deleting content, and making queries.</th></tr>\n \n <tr align=left><th><img src=\"Images/new.jpg\"><a href=\"RDocbook\">RDocbook</a>\n\t</th><th>Code for working with R and Docbook documents.\n\t These are external facilities (from R) such as\n\t Emacs bindings and shell scripts that facilitate authoring\n\tand processing Docbook documents with R code.</th></tr>\n \n <tr align=left><th><img src=\"Images/new.jpg\"><a href=\"XMLRPC\">XMLRPC</a>\n\t</th><th>A package that provides basic XML-RPC client\n\tfacilities for accessing remote methods in XML-RPC\n\tservers.</th></tr>\n\n <tr align=left><th><img src=\"Images/new.jpg\"><a href=\"RExcelXML\">RExcelXML</a>\n\t</th><th>Facilities for working with .xlsx documents,\n\ti.e. Excel 2007.</th></tr>\n\n\n <tr align=left><th><img src=\"Images/new.jpg\"><a href=\"RUbigraph\">RUbigraph</a>\n\t</th><th>A package that provides an interface to Ubigraph,\n\t an Open GL application for rendering interactive graphs\n\t(node & edges). The connection is via XML-RPC.</th></tr>\t\n \n <tr align=left><th><img src=\"Images/new.jpg\"><a href=\"RKML\">RKML</a>\n\t</th><th>A package that provides high-level functions\n\tfor creating KML displays for Google Earth from within R,\n\t using spatial temporal data.</th></tr>\n \n <tr align=left><th><img src=\"Images/new.jpg\"><a href=\"R2GoogleMaps\">R2GoogleMaps</a>\n\t</th><th>A package that allows R programmers to create HTML\n\tand JavaScript code to display content via Google Maps in a\n\tWeb Browser</th></tr>\n \n <tr align=left><th><img src=\"Images/new.jpg\"><a href=\"RJavaScript\">RJavaScript</a>\n\t</th><th>A package that converts R code to JavaScript to help\n\tcreating JavaScript content for Web pages, Flash applications,\n\tSVG displays, ...</th></tr>\n \n <tr align=left><th><img src=\"Images/new.jpg\"><a href=\"RAmazonS3\">RAmazonS3</a>\n\t</th><th>Package that allows R programmers to interact with\n\tAmazon's S3 storage server, uploading and downloading and\n\tgenerally managing files.</th></tr>\n \n <tr align=left><th><img src=\"Images/new.jpg\"><a href=\"RRuby\">RRuby</a>\n\t</th><th>Package that allows R programmers to call Ruby from R</th></tr>\n\n <tr align=left><th><img src=\"Images/new.jpg\"><a href=\"Rcrypt\">Rcrypt</a>\n\t</th><th>Interface to the crypt C routine for encrypting passwords.</th></tr>\n\n <tr align=left><th><img src=\"Images/new.jpg\"><a href=\"RGraphicsDevice\">RGraphicsDevice</a>\n\t</th><th>Package that allows R programmers to implement R graphics devices entirely within R code</th></tr>\n <tr align=left><th><img src=\"Images/new.jpg\"><a href=\"FlashMXML\">FlashMXML</a>\n\t</th><th>a prototype of an R graphics device that generates Flash/Flex content.</th></tr>\n\n <tr align=left><th><img src=\"Images/new.jpg\"><a href=\"XMLSchema\">XMLSchema</a>\n\t</th><th>A package for reading and processing XML schema in R</th></tr>\n\n <tr align=left><th><img src=\"Images/new.jpg\"><a href=\"SSOAP\">SSOAP</a>\n\t</th><th>A package for R client-side access to SOAP-based Web services </th></tr>\n\n<tr>\n <tr align=left><th>\n <a href=\"http://www.omegahat.net/RSXML\">XML</a>\n </th>\n <th>Numerous updates for parsing and generating XML content. </th>\n</tr> \n\n<tr>\n <tr align=left><th>\n <a href=\"http://www.omegahat.net/RCurl\">RCurl</a>\n </th>\n <th>Updates for new versions of libcurl and the many new options.</th>\n</tr> \n\n <tr align=left><th><a href=\"Rcompression/index.html\">Rcompression package</a>\n\t</th><th> Interface to zlib and bzip2 libraries for performing in-memory compression and decompression\n in R. This is useful when receiving or sending contents to remote servers, e.g. Web services, HTTP requests via RCurl.\n</th></tr>\n\n <tr align=left><th><a href=\"Sxslt/index.html\">Sxslt package</a>\n\t</th><th>Updates to the bi-directional interface between R and\n\tlibxslt.\n In particular, the use of XSLT from within R is greatly\n\tenhanced with much greater control of registering functions, \n parameters, etc.\n</th></tr>\n<tr>\n <tr align=left><th>\n <a href=\"http://www.omegahat.net/RSJava\">SJava</a>\n </th>\n <th>\n Bug fixes and new features for converters.\n And removed the dependency on an old version of ANTLR.\n </th>\n</tr> \n\n\n\n<tr>\n <tr align=left><th>\n <a href=\"http://www.omegahat.net/Rexif\">Rexif</a>\n </th>\n <th>\n An R package for reading camera information\n and other meta-data from images taken with digital cameras.\n </th>\n</tr> \n\n\n\n<tr>\n <tr align=left><th>\n <a href=\"http://www.omegahat.net/CGIwithR\">CGIwithR</a>\n </th>\n <th>\n Extensions to David Firth's original CGIwithR package.\n I am now maintaining that package after David's \n good work.\n </th>\n</tr> \n\n\n<tr>\n <tr align=left><th>\n <a href=\"http://www.omegahat.net/Aspell\">Aspell</a>\n </th>\n <th>\n An interface to the GNU aspell library for checking\n the spelling of words and entire documents.\n </th>\n</tr> \n\n \n<tr>\n <tr align=left><th>\n <a href=\"http://www.omegahat.net/RDCOMEvents\">RDCOMEvents</a>\n </th>\n <th>\n This package provides a facility to listen for DCOM events\n from applications such as Excel and Word and others and to \n process them with R functions.\n </th>\n</tr> \n\n\n <tr align=left><th><img src=\"Images/new.jpg\">\n <a href=\"http://www.omegahat.net/Combinations\">Combinations</a>\n </th>\n <th>\n This package is used for generating the combinations for choosing r\n items from a set of n items but does it in C code.\n When n gets large, the package provides a mechanism for dealing with each combination\n as it is generated so that one does not have to hold the entire collection around\n and operate on them after creating the entire collection.\n This makes computations feasible for very large numbers of combinations. It does not \n necessarily make them fast!\n The package is also a good example of the basics of calling R functions from C code.\n It also illustrates aspects of adapting code for a stand-alone application to be used within\n an interactive language like R/S-Plus, Python, Perl or Matlab/Octave.\n </th>\n</tr> \n\n\n\n<tr>\n <tr align=left><th>\n <a href=\"http://www.omegahat.net/RMatlab\">RMatlab</a>\n </th>\n <th>\n An interface between R and Matlab.\n One can call R functions from Matlab code,\n and Matlab functions from R code\n using the syntax of the local programming language.\n </th>\n</tr> \n \n<tr>\n <tr align=left><th><img src=\"Images/new.jpg\">\n <a href=\"http://www.omegahat.net/RCurl\">RCurl</a>\n </th>\n <th>\n A package for performing HTTP requests\n using the C-level cURL library.\n This is a very flexible, customizable way to\n handle HTTP-based communications from within R\n in a higher-level and more extensive way than\n is directly available via socket connections.\n This is used by the <a href=\"http://www.omegahat.net/SSOAP\">SSOAP\n package</a>\n and a forthcoming package for HTML form submissions.\n </th>\n</tr> \n\n \n<tr>\n <tr align=left><th>\n <a href=\"http://www.omegahat.net/Rstem\">Rstem</a>\n </th>\n <th>\n A package for performing word stemming\n for use in text analysis.\n This is an interface to Martin Porter's\n <a href=\"http://snowball.tartarus.org\">Snowball</a>\n generated C code for performing\n stemming. The package has\n support for several languages\n and is exensible.\n </th>\n</tr> \n \n <tr>\n <tr align=left><th>\n <a href=\"http://www.omegahat.net/IDocs\">IDocs</a> \n </th> \n <th>\n An experimental package for interactive documents\n created in XML and rendered using\n viewHtml in <a href=\"RGtkViewers\">RGtkViewers</a>.\n This connects the different plugins in a document\n and across documents using nested environments and\n a stack that pushes and pops environments as they\n are entered and exited.\n </th></tr>\n\n <tr>\n <tr align=left><th>\n <a href=\"http://www.omegahat.net/OmegahatXSL\">XSL files\n\tfor Omegahat</a> \n </th> \n <th>\n This is a collection of support XSL, DTD and CSS files\n for Omegahat. The suite allows thefiles to be used via <i>catalogs</i>.\n </th></tr>\n \n\n \n <tr>\n <tr align=left><th>\n <a href=\"http://www.omegahat.net/SASXML\">SASXML</a> \n </th> \n <th>\n An experimental package for reading XML data in\n the Open Information Model (OIM) format,\n e.g. SAS XML files.\n This might be generalized to have functionality\n for reading PMML (Predictive Model Markup Language).\n </th></tr>\n\n \n <tr>\n <tr align=left><th>\n <a href=\"http://www.omegahat.net/RDCOMServer\">RDCOMServer</a> \n </th> \n <th>\n A rich, extensible and flexible mechanism for defining\n and implementing COM classes and objects\n entirely within the S language.\n This allows one to make S functions and objects\n accessible to other applications and programming languages\n without people having to know the S language.\n This provids not just access to the R evaluator, but\n arbitrary S objects.\n </th></tr>\n\n\n <tr>\n <tr align=left><th>\n <a href=\"http://www.omegahat.net/RDCOMClient\">RDCOMClient</a> \n </th> \n <th>\n A mechanism that allows S users to create and use COM objects\n directly within an R session.\n </th></tr>\n \n\n \n <tr>\n <tr align=left><th>\n <a href=\"http://www.omegahat.net/RGnomePrint\">RGnomePrint</a> \n </th> \n <th>\n An R interface to libgnomeprint, the Gnome printing library that\n provides facilities for controlling printing from Gtk widgets,\n previewing, and generating Postscript, Windows metafiles (and\n potentially PDF.)\n </th></tr>\n\n \n <tr>\n <tr align=left><th>\n <a href=\"http://www.omegahat.net/GccTranslationUnit\">GccTranslationUnit</a> \n </th> \n <th>\n A Python module for obtaining reflectance information from C/C++\n source code. This can be used to programmatically generate S bindings\n to arbitrary routines, classes and data structures\n in native code and also build the registration\n table for routines to be accessed from R.\n We can also use this to compute on C/C++ code,\n e.g. determining call graphs, free variables. \n </th></tr>\n\n \n <tr>\n <tr align=left><th>\n <a href=\"http://www.omegahat.net/SWinTypeLibs\">SWinTypeLibs</a> \n </th> \n <th>\n Facilities for reading type information from DLLs, type\n libraries or (D)COM objects directly in R.\n This can be used to programmatically generate interfaces\n to C/C++ code.\n </th></tr>\n\n <tr>\n <tr align=left><th>\n <a href=\"http://www.omegahat.net/SWinRegistry\">SWinRegistry</a> \n </th>\n <th>\n Read and write access to the Windows registry.\n For Windows platform only.\n </th></tr> \n \n \n \n\n \n <tr>\n <tr align=left><th><a\n\thref=\"RGtkExtra/index.html\">S language binding\n\tto the gtk+extra library.\n\t</a>\n\t</th><th>\n\t Access to the widgets in the gtk+extra library,\n\t i.e. the data-grid/sheet display,\n\t directory list, icon list and file list.\n\t</th></tr>\n\n <tr>\n <tr align=left><th><a\n\thref=\"RGdkPixbuf/index.html\">S language binding\n\tto the Gdk pixbuf library.\n\t</a>\n\t</th><th>\n\t Access to the Pixbuf library for reading\n\t images into R to be used in RGtk\n\t and associated packages.\n\t</th></tr>\t\n\n <tr>\n <tr align=left><th><a\n\thref=\"RGtkGlade/index.html\">S language binding\n\tto the Glade library.\n\t</a>\n\t</th><th>\n\t S language facilities for instantiating\n\t GUIs constructed interactively via the\n\t glade GUI builder.\n\t This can register S functions or expressions, or C routines specified\n\t as callbacks in the glade interface.\n\t</th></tr>\n\n\n <tr>\n <tr align=left><th><a\n\thref=\"REventLoop/index.html\">\n\tA plugin event-loop for R.\n\t</a>\n\t</th><th>\n A facility that allows one to dynamically\n\t specify a new event loop to use within\n\t an R session. This allows one to use\n\t Gtk or Tk event loop\n\t to drive R. One can write methods\n\t for other toolkits such as Carbon, Qt, etc.\n\t</th></tr>\t\t\n \n\t\n <tr>\n <tr align=left><th><a\n\thref=\"RGtkBindingGenerator/index.html\">S language binding\n\tgenerator for Gtk-based libraries.\n\t</a>\n\t</th><th>A package that creates S and C code to\n\tinterface to arbitrary Gtk-based C libraries and Gtk classes\n\t such as Gtk itself, Zvt, gtk+-extra, gtkhtml, embedded\n\t Mozilla, etc.\n\t</th></tr>\n\n <tr align=left><th><img src=\"Images/new.jpg\"><a href=\"RSXML/index.html\">R/SPlus XML Parsing\n\tpackage</a>\n\t</th><th>Parse and create XML documents within the S\n\t language.</th></tr>\n\n <tr align=left><th><a href=\"RGtk/index.html\">RGtk package </a>\n\t</th><th>Bindings for programming GUIs using Gtk and extended\n\t widget sets.</th></tr>\t \n \n <tr align=left><th><a href=\"RXLisp/index.html\">An R-XLisp\n\tinterface </a>\n\t</th><th>Interface between R and XLisp allowing R code to call\n\t XLisp functions.</th></tr> \n <tr align=left><th><a href=\"RObjectTables/index.html\">RObjectTables\n\tpackage </a>\n\t</th><th>Framework for user-defined search path elements,\n\t alternative automatic serialization, \n\t external variables, etc.</th></tr>\n <tr align=left><th><a href=\"RGtkViewers/index.html\">RGtkViewers package</a>\n\t</th><th>A collection of GUI tools for viewing different\n\t information, built using RGtk.</th></tr> \t\t \n <tr align=left><th><a\n href=\"RSMethods/index.html\">RSMethods</a></th>\n <th>Formal S classes and methods for R.</th>\n <tr></tr>\n <tr align=left><th><img src=\"Images/new.jpg\"><a href=\"RSPython/index.html\">RSPython</a></th>\n <th>a bi-directional interface allowing calling\n R functions from Python code and accessing Python\n objects, classes and functions from R sessions.</th></tr>\n\n <tr align=left><th><a href=\"RSPerl/index.html\">RSPerl</a></th>\n <th>a bi-directional interface allowing calling\n R functions from Perl code and accessing Perl\n objects, classes and routies from R sessions.</th></tr> \n \n <tr align=left><th><a\n\thref=\"RGnumeric/index.html\">RGnumeric</a></th>\n\t<th>an R plugin for Gnumeric that allows Gnumeric\n\tusers to call R functions from within a spreadsheet,\n\tand for R programmers to access Gnumeric sheets\n\tand their contents from within R functions.</th>\n </tr>\n \n \n <tr align=left><th><a href=\"SXalan\">R embedded in Xalan, the XSL\n translator</a></th><th>Use R to generate output in and control\n\t transformation of XML documents using XSL functions</th></tr>\n <tr align=left><th><a href=\"SNetscape\">R Plugin for\n Netscape</th><th>\n\t A plugin for Netscape that allows one to call R from\n\t JavaScript and JavaScript from R.\n\t </th></tr>\n </table> \n</center>\n\nThese are just some of a <a href=\"download/R/packages\">complete collection of Omegahat packages for R and S-Plus</a>.\n \n<center><h3>\n</h3></center>\n\n\n<hr width=50%>\n\nOmega is a joint project with the goal of providing a variety of\n<a href=\"License.html\">open-source</a> software for statistical\napplications.\nThe Omega project began in July, 1998, with discussions among\ndesigners responsible for three current statistical languages (S, R,\nand Lisp-Stat), with the idea of working together on new directions\nwith special emphasis on web-based software, Java, the Java\nvirtual machine, and distributed computing.\nWe encourage participation by anyone wanting to extend\ncomputing capabilities in one of the existing languages, to those\ninterested in distributed or web-based statistical software, and to\nthose interested in the design of new statistical languages.\n<p>\nSoftware is being developed for the project; we refer to it as the <FONT\nCLASS=Omegahat>omegahat</FONT> software, to emphasize in typical\nstatistical terminology that we are starting on a process of\napproximating the ideal software through many iterations, among other\n<a href=\"Name.html\">reasons</a>.\n<p>\n\nFollow the pointers below to find out more about the omegahat software.\n<ul>\n <li> <a href=\"#Levels\">Organization of the Software.</a>\n <li> <a href=\"#Documentation\">Documentation. </a>\n <li>\n <a href=\"http://www.omegahat.net/mailman/listinfo\">Mailing Lists.</a>.\n<li> <a href=\"download/index.html\">Source & Binary\n Distributions.</a>\n<li><a href=\"#Related\">Related Software.</a>\n<p>\n</ul>\n<p>\n\n<h2><a href=\"philosophy.html\">Philosophy</a></h2>\n\n<h2><a name=\"Related\">Related Software</a></h2><p>\nSome of the many areas in which Java packages may be of value to statistics\ninclude:\n<ul>\n<li>the JDBC database connectivity software, providing a general interface to database\nsystems, extended by some facilities typically needed in\nstatistical applications;\n<li> numerical computations for linear algebra (e.g.,\nwork is currently proceeding on two Java matrix\nand linear algebra packages called\n<a href=\"http://math.nist.gov/javanumerics/jama/\">JAMA</a>\n and\n<a href=\"http://nicewww.cern.ch/~hoschek/colt/index.htm\">COLT</a>\n);\n<li>processing and analysis of images---can we combine\ncomputational techniques with the flexibility of graphics\nand user interface tools in Java;\n<li>graphics---possible new models for statistical graphics;\n<li>dynamic reporting -- the Java packages supporting \n<a href=\"http://www.w3.org/XML\">XML</a>, lightweight components\n(<a href=\"http://www.javasoft.com/products/jfc/tsc/index.html\">Swing/Java Foundation Classes</a>), HTML viewers\nand network-distributed computations make the Omega\nenvironment useful for generating reports with live objects.\n</ul>\nIn many of these areas, there have been questions about the speed\nattainable by Java. Investigating such questions is part of what we\nwant to do.\n\n<h2><a name=toc3>Policy</a></h2>\n<p>\nSpecific goals and procedures are still being developed, but the current\npolicy includes the following.\n<ul>\n<li>The code developed will be created and offered as open-source\nsoftware, under a <a href=\"License.html\">public license</a> similar to the licenses used for\nLinux, Netscape, and R.\n<li>Contributions are encouraged in the form of software compatible\nwith the three-level structure shown above. This includes but is not restricted\nto: Java packages; software in other languages, if it can be structured as a module\ndefined in CORBA; software for the interactive languages built in\nOmega.\n<li>We anticipate organizing software into layers, including at\nleast two: a general contributed software layer and a central layer for\ncore Omega facilities.\n<li>We have established a number of\n<a href=\"http://www.omegahat.net/mailman/listinfo/\">mailing\nlists</a> for exchange of ideas. These include lists for general\ndiscussion, for Omega packages, for interactive languages using\nOmega, and for technical issues about Omega itself.\n</ul>\n<p>\n<h2><a name=toc4>Membership</a></h2>\n<p>\nThe current core group of Omega developers includes:\n<center>\n<table border=\"1\">\n<tr>\n<th align=left><font size=-1><a href=\"http://www.stat.wisc.edu/~bates/\">Douglas Bates</a></font></th>\n<th align=left><font size=-1><a href=\"http://cm.bell-labs.com/stat/jmc/\">John Chambers</a></font> </th>\n<th align=left><font size=-1><a href=\"http://www.public.iastate.edu/~dicook\">Di Cook</a></font> </th> \n</tr>\n<tr>\n<th align=left><font size=-1><a href=\"http://www.biostat.ku.dk/~pd\">Peter Dalgaard</a></font></th>\n<th align=left><font size=-1><a href=\"http://www.stat.auckland.ac.nz/rgentlem/rgentlem.html\">Robert Gentleman</a></font> </th>\n<th align=left><font size=-1><a href=\"http://www.ci.tuwien.ac.at/~hornik/\">Kurt Hornik</a></font></th>\n</tr>\n<tr>\n<th align=left><font size=-1><a href=\"http://www.stat.auckland.ac.nz/PEOPLE/ihaka/rihaka.html\">Ross Ihaka</a></font> </th>\n<th align=left><font size=-1><a href=\"http://www.ci.tuwien.ac.at/~leisch/\">Friedrich Leisch</a></font> </th>\n<th align=left><font size=-1><a href=\"http://www.biostat.washington.edu/~thomas/\">Thomas Lumley</a></font></th>\n<tr>\n<th align=left><font size=-1><a href=\"http://stat.ethz.ch/~maechler/\">Martin Mächler</a></font></th>\n<th align=left><font size=-1><a href=\"http://sirio.stat.unipd.it/\">Guido Masarotto</a></font></th> \n<th align=left><font size=-1><a href=\"http://www.stat.auckland.ac.nz/PEOPLE/paul/paul.shtml\">Paul Murrell</a></font></th>\n</tr>\n<tr>\n<th align=left><font size=-1><a href=\"http://www-stat.stanford.edu/~naras/\">Balasubramanian Narasimhan</a></font></th> \n<th align=left><font size=-1><a href=\"http://www.stats.ox.ac.uk/~ripley/\">Brian Ripley</a></font></th>\n<th align=left><font size=-1><a href=\"http://www.statlab.uni-heidelberg.de/people/sawitzki/\">Günther Sawitzki</a></font></th>\n</tr>\n<tr>\n<th align=left><font size=-1><a href=\"http://www.stat.ucdavis.edu/~duncan/\">Duncan Temple Lang</a></font></th>\n<th align=left><font size=-1><a href=\"http://stat.umn.edu/~luke/\">Luke Tierney</a></font> </th>\n<th align=left><font size=-1><a href=\"http://www.cmis.csiro.au/bill.venables/\">Bill Venables</a></font></th>\n</tr>\n</table>\n</center>\n<p>\nParticipation in the general and special-purpose software work of the\nOmega project is open to anyone interested in our goals.\nThe <a href=\"http://www.omegahat.net/mailman/listinfo/\">mailing\nlists</a> are a good place to start; in particular\n<a name=\"NWD1\"></a>\n<a href=\"mailto:omega-devel@omegahat.net\">omega-devel@omegahat.net</a>\nis a list for discussion of the general development of software\nfor the project.\nThe success of the Omega project is highly dependent on the\ninvolvement of a wide range of participants in many different areas.\nWe can all benefit.\n<p>\n\n<h2>The Omega Project Umbrella</h2>\n\nAs well as developing software ourselves,\nthe project also hopes to support other development efforts.\nTo this end, Doug Bates has generously allowed his machine to be used\nfor distributing code for different projects.\nThese projects include.\n<ul>\n <li> <a href=\"contrib/RS-DBI/index.html\">RS-DBI: Database\n connectivity for both R and S.</a>\n</ul>\nMore general details are <a href=\"contrib/index.html\">also available</a>.\n\n</body> </html>\n"
# Arrange to read the header of the response from the HTTP server as
# a separate "stream". Then we can break it into name-value
# pairs. (The first line is the HTTP/1.1 200 Ok or 301 Moved Permanently
# status line)
if(omegahatExists) withAutoprint({
h = basicTextGatherer()
txt = getURL("https://www.omegahat.net/RCurl/index.html",
header= TRUE, headerfunction = h$update,
httpheader = c(Accept="text/html", Test=1), verbose = TRUE)
print(paste(h$value(NULL)[-1], collapse=""))
con <- textConnection(paste(h$value(NULL)[-1], collapse=""))
read.dcf(con)
close(con)
})
#> > h = basicTextGatherer()
#> > txt = getURL("https://www.omegahat.net/RCurl/index.html", header = TRUE,
#> + headerfunction = h$update, httpheader = c(Accept = "text/html", Test = 1), verbose = TRUE)
#> > print(paste(h$value(NULL)[-1], collapse = ""))
#> [1] "Date: Wed, 29 Oct 2025 18:26:39 GMT\r\nServer: Apache/2.4.37 (Rocky Linux) OpenSSL/1.1.1k mod_fcgid/2.3.9\r\nLast-Modified: Sat, 21 Jun 2014 03:27:39 GMT\r\nETag: \"10bc-4fc50312600c0\"\r\nAccept-Ranges: bytes\r\nContent-Length: 4284\r\nContent-Type: text/html; charset=UTF-8\r\n\r\n"
#> > con <- textConnection(paste(h$value(NULL)[-1], collapse = ""))
#> > read.dcf(con)
#> Date
#> [1,] "Wed, 29 Oct 2025 18:26:39 GMT"
#> Server
#> [1,] "Apache/2.4.37 (Rocky Linux) OpenSSL/1.1.1k mod_fcgid/2.3.9"
#> Last-Modified ETag Accept-Ranges
#> [1,] "Sat, 21 Jun 2014 03:27:39 GMT" "\"10bc-4fc50312600c0\"" "bytes"
#> Content-Length Content-Type
#> [1,] "4284" "text/html; charset=UTF-8"
#> > close(con)
# Test the passwords.
if(omegahatExists) withAutoprint({
x = getURL("https://www.omegahat.net/RCurl/testPassword/index.html", userpwd = "bob:duncantl")
# Catch an error because no authorization
# We catch the generic HTTPError, but we could catch the more specific "Unauthorized" error
# type.
x = tryCatch(getURLContent("https://www.omegahat.net/RCurl/testPassword/index.html"),
HTTPError = function(e) {
cat("HTTP error: ", e$message, "\n")
})
})
#> > x = getURL("https://www.omegahat.net/RCurl/testPassword/index.html", userpwd = "bob:duncantl")
#> > x = tryCatch(getURLContent("https://www.omegahat.net/RCurl/testPassword/index.html"),
#> + HTTPError = function(e) {
#> + cat("HTTP error: ", e$message, "\n")
#> + })
#> HTTP error: Unauthorized
#>
if (FALSE) { # \dontrun{
# Needs specific information from the cookie file on a per user basis
# with a registration to the NY times.
x = getURL("https://www.nytimes.com",
header = TRUE, verbose = TRUE,
cookiefile = "/home/duncan/Rcookies",
netrc = TRUE,
maxredirs = as.integer(20),
netrc.file = "/home2/duncan/.netrc1",
followlocation = TRUE)
} # }
if(interactive() && omegahatExists) {
d = debugGatherer()
x = getURL("https://www.omegahat.net", debugfunction = d$update, verbose = TRUE)
d$value()
}
#############################################
# Using an option set in R
if(interactive() && omegahatExists) {
opts = curlOptions(header = TRUE, userpwd = "bob:duncantl", netrc = TRUE)
getURL("https://www.omegahat.net/RCurl/testPassword/index.html", verbose = TRUE, .opts = opts)
# Using options in the CURL handle.
h = getCurlHandle(header = TRUE, userpwd = "bob:duncantl", netrc = TRUE)
getURL("https://www.omegahat.net/RCurl/testPassword/index.html", verbose = TRUE, curl = h)
}
# Use a C routine as the reader. Currently gives a warning.
if(interactive() && omegahatExists) {
routine = getNativeSymbolInfo("R_internalWriteTest", PACKAGE = "RCurl")$address
getURL("https://www.omegahat.net/RCurl/index.html", writefunction = routine)
}
# Example
if(interactive() && omegahatExists) {
uris = c("https://www.omegahat.net/RCurl/index.html",
"https://www.omegahat.net/RCurl/philosophy.xml")
txt = getURI(uris)
names(txt)
nchar(txt)
txt = getURI(uris, async = FALSE)
names(txt)
nchar(txt)
routine = getNativeSymbolInfo("R_internalWriteTest", PACKAGE = "RCurl")$address
txt = getURI(uris, write = routine, async = FALSE)
names(txt)
nchar(txt)
# getURLContent() for text and binary
x = getURLContent("https://www.omegahat.net/RCurl/index.html")
class(x)
x = getURLContent("https://www.omegahat.net/RCurl/data.gz")
class(x)
attr(x, "Content-Type")
x = getURLContent("https://www.omegahat.net/Rcartogram/demo.jpg")
class(x)
attr(x, "Content-Type")
curl = getCurlHandle()
dd = getURLContent("https://www.omegahat.net/RJSONIO/RJSONIO.pdf",
curl = curl,
header = dynCurlReader(curl, binary = TRUE,
value = function(x) {
print(attributes(x))
x}))
}
# FTP
# Download the files within a directory.
if(interactive() && url.exists('ftp://ftp.wcc.nrcs.usda.gov')) {
url = 'ftp://ftp.wcc.nrcs.usda.gov/data/snow/snow_course/table/history/idaho/'
filenames = getURL(url, ftp.use.epsv = FALSE, dirlistonly = TRUE)
# Deal with newlines as \n or \r\n. (BDR)
# Or alternatively, instruct libcurl to change \n's to \r\n's for us with crlf = TRUE
# filenames = getURL(url, ftp.use.epsv = FALSE, ftplistonly = TRUE, crlf = TRUE)
filenames = paste(url, strsplit(filenames, "\r*\n")[[1]], sep = "")
con = getCurlHandle( ftp.use.epsv = FALSE)
# there is a slight possibility that some of the files that are
# returned in the directory listing and in filenames will disappear
# when we go back to get them. So we use a try() in the call getURL.
contents = sapply(filenames[1:5], function(x) try(getURL(x, curl = con)))
names(contents) = filenames[1:length(contents)]
}