Querying data from a website using R and GET function(使用R和GET函数从网站查询数据)

本文介绍了使用R和GET函数从网站查询数据的处理方法，对大家解决问题具有一定的参考价值，需要的朋友们下面随着小编来一起学习吧！

问题描述

我对网络抓取非常陌生，我需要下载在进行查询后出现几个老套的数据。这意味着我需要在第一页中填写两个字段，然后在粗体文本上循环，然后标识一个大写的数据表，并下载它。

我首先使用GET函数，并将所需的姓名作为列表添加到"Query"参数中。然而，尽管我是一个老R用户，我甚至无法破译我收到的错误。

GET("http://apps.kew.org/wcsp/advsearch.do;jsessionid=15925570A99B794122939889DE7DCDBC",path = "search", query =list(Genus="Imperata",Species="cylindrica"))


Response[http://apps.kew.org/search;jsessionid=15925570A99B794122939889DE7DCDBC?      Genus=Imperata&Species=cylindrica]

日期：2016-04-18 18：29
状态：404
Content-Type：Text/html；Charset=iso-8859-1
尺寸：445 B

未找到404

未找到

未找到请求的url/search；jsessionid=15925570A99B794122939889DE7DCDBC...

此外，403禁用
尝试使用ErrorDocument处理请求时出错。

Apps.kew.org端口80处的Apache2.2.3(Red Hat)服务器

推荐答案

它可能无法工作，因为它是POST请求与GET请求(您可以使用我的curlconverter包来帮助处理这些隐藏的API，btw)：

library(httr)
library(rvest)

res <- httr::POST(url = "http://apps.kew.org/wcsp/advsearch.do", 
           body = list(page = "advancedSearch", 
                       AttachmentExist = "", 
                       family = "", 
                       placeOfPub = "", 
                       genus = "Imperata", 
                       yearPublished = "", 
                       species = "cylindrica", 
                       author = "", 
                       infraRank = "", 
                       infraEpithet = "", 
                       selectedLevel = "cont"), 
           encode = "form") 


pg <- content(res, as="parsed")

html_text(html_nodes(pg, "a.onwardnav"))

##  [1] "Imperata cylindrica (L.) P.Beauv., Ess. Agrostogr.: 165 (1812)."                                                
##  [2] "Imperata cylindrica var. africana (Andersson) C.E.Hubb., Joint Publ. Imp. Agric. Bur. 7: 10 (1944)."            
##  [3] "Imperata cylindrica var. condensata (Steud.) Hack., Anales Mus. Nac. Hist. Nat. Buenos Aires 21: 9 (1911)."     
##  [4] "Imperata cylindrica var. europaea (Andersson) Asch. & Graebn., Syn. Mitteleur. Fl. 2(1): 37 (1898)."            
##  [5] "Imperata cylindrica subsp. koenigii (Retz.) Masamura & Yanagih., Trans. Nat. Hist. Soc. Formosa 31: 326 (1941)."
##  [6] "Imperata cylindrica subvar. koenigii (Retz.) T.Durand & Schinz, Consp. Fl. Afric. 5: 694 (1894)."               
##  [7] "Imperata cylindrica var. koenigii (Retz.) Pilg., Fragm. Fl. Philipp. 1: 137 (1904)."                            
##  [8] "Imperata cylindrica var. latifolia (Hook.f.) C.E.Hubb., Joint Publ. Imp. Agric. Bur. 7: 14 (1944)."             
##  [9] "Imperata cylindrica var. major (Nees) C.E.Hubb., Grasses Mauritius: 96 (1940)."                                 
## [10] "Imperata cylindrica var. mexicana (Rupr. ex Galeotti) D.B.Ward, Novon 14: 368 (2004)."                          
## [11] "Imperata cylindrica f. pallida Honda, J. Fac. Sci. Univ. Tokyo, Sect. 3, Bot. 3: 374 (1930)."                   
## [12] "Imperata cylindrica var. parviflora Batt. & Trab., Bull. Soc. Bot. France 53: 32 (1906)."                       
## [13] "Imperata cylindrica var. pedicellata (Steud.) Debeaux, Actes Soc. Linn. Bordeaux 32: 52 (1878)."                
## [14] "Imperata cylindrica var. thunbergii (Retz.) T.Durand & Schinz, Consp. Fl. Afric. 5: 693 (1894), nom. superfl."  

lnks <- html_attr(html_nodes(pg, "a.onwardnav"), "href")

res2 <- GET(sprintf("http://apps.kew.org%s", lnks[1]))
pg2 <- content(res2, as="parsed")

trimws(gsub("[[:space:]]+", " ", html_text(html_nodes(pg2, "th + td"))))

## [1] "Medit. to Africa and Afghanistan 12 BAL COR FRA POR SAR SPA 13 ALB BUL GRC ITA KRI SIC TUE YUG 20 ALG EGY LBY MOR TUN 21 CNY CVI MDR 22 BEN BKN GAM GHA GNB GUI IVO LBR MLI NGA NGR SEN SIE TOG 23 BUR CAF CMN CON EQG GAB GGI RWA ZAI 24 CHA ETH SOC SUD 25 KEN TAN UGA 26 ANG MLW MOZ ZAM ZIM 27 BOT CPP LES NAM NAT OFS SWZ TVL 29 COM MAU MDG (32) kaz kgz tkm tzk uzb (33) ncs tcs 34 AFG CYP EAI IRN IRQ LBS PAL SIN TUR 35 KUW OMA? SAU YEM (36) chc chh chi chm chn chs cht chx (38) jap kor nns oga tai (40) ass ban ehm ind nep pak srl whm (41) and cbd lao mya ncb scs tha vie (42) bor cki jaw lsi mly mol phi sul sum xms (43) bis nwg sol (50) nfk (51) nzn (60) fij nwc sam ton van wal (62) mrn (73) ore (77) tex (78) ala fla geo lou msi sca vrg (79) mxs mxt"
## [2] "Hemicr. or rhizome geophyte"    
## [3] "Poaceae"                                     
## [4] "W.D.Clayton, R.Govaerts, K.T.Harman, H.Williamson & M.Vorontsova"

这篇关于使用R和GET函数从网站查询数据的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持编程学习网！