常见的HTTP客户端和代理

我正在使用apache的常见httpclient库。 是否可以通过代理发出HTTP请求？ 更具体地说，我需要使用代理列表来处理multithreadingPOST请求（现在我正在使用单线程GET请求进行测试）。

我试过用：

httpclient.getHostConfiguration().setProxy("67.177.104.230", 58720); 

我得到该代码的错误：

 21.03.2012. 20:49:17 org.apache.commons.httpclient.HttpMethodDirector executeWithRetry INFO: I/O exception (java.net.ConnectException) caught when processing request: Connection refused: connect 21.03.2012. 20:49:17 org.apache.commons.httpclient.HttpMethodDirector executeWithRetry INFO: Retrying request 21.03.2012. 20:49:19 org.apache.commons.httpclient.HttpMethodDirector executeWithRetry INFO: I/O exception (java.net.ConnectException) caught when processing request: Connection refused: connect 21.03.2012. 20:49:19 org.apache.commons.httpclient.HttpMethodDirector executeWithRetry INFO: Retrying request 21.03.2012. 20:49:21 org.apache.commons.httpclient.HttpMethodDirector executeWithRetry INFO: I/O exception (java.net.ConnectException) caught when processing request: Connection refused: connect 21.03.2012. 20:49:21 org.apache.commons.httpclient.HttpMethodDirector executeWithRetry INFO: Retrying request org.apache.commons.httpclient.ProtocolException: The server xxxxx failed to respond with a valid HTTP response at org.apache.commons.httpclient.HttpMethodBase.readStatusLine(HttpMethodBase.java:1846) at org.apache.commons.httpclient.HttpMethodBase.readResponse(HttpMethodBase.java:1590) at org.apache.commons.httpclient.HttpMethodBase.execute(HttpMethodBase.java:995) at org.apache.commons.httpclient.ConnectMethod.execute(ConnectMethod.java:144) at org.apache.commons.httpclient.HttpMethodDirector.executeConnect(HttpMethodDirector.java:495) at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:390) at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:170) at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:396) at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:324) at test.main(test.java:42) 

当我删除该行时，一切都按预期正常运行。

Answer 1

对于httpclient 4.1.x，您可以像这样设置代理（取自此示例 ）：

  HttpHost proxy = new HttpHost("127.0.0.1", 8080, "http"); DefaultHttpClient httpclient = new DefaultHttpClient(); try { httpclient.getParams().setParameter(ConnRoutePNames.DEFAULT_PROXY, proxy); HttpHost target = new HttpHost("issues.apache.org", 443, "https"); HttpGet req = new HttpGet("/"); System.out.println("executing request to " + target + " via " + proxy); HttpResponse rsp = httpclient.execute(target, req); ... } finally { // When HttpClient instance is no longer needed, // shut down the connection manager to ensure // immediate deallocation of all system resources httpclient.getConnectionManager().shutdown(); } 

Answer 2

以下是如何使用最新版本的HTTPClient （4.3.4）

  CloseableHttpClient httpclient = HttpClients.createDefault(); try { HttpHost target = new HttpHost("localhost", 443, "https"); HttpHost proxy = new HttpHost("127.0.0.1", 8080, "http"); RequestConfig config = RequestConfig.custom() .setProxy(proxy) .build(); HttpGet request = new HttpGet("/"); request.setConfig(config); System.out.println("Executing request " + request.getRequestLine() + " to " + target + " via " + proxy); CloseableHttpResponse response = httpclient.execute(target, request); try { System.out.println("----------------------------------------"); System.out.println(response.getStatusLine()); EntityUtils.consume(response.getEntity()); } finally { response.close(); } } finally { httpclient.close(); } 

Answer 3

从Apache HTTPComponents 4.3.x开始，HttpClientBuilder类从系统属性http.proxyHost和http.proxyPort设置代理默认值，否则您可以使用setProxy方法覆盖它们。

Answer 4

虽然这个问题很老，但我看到的还没有确切的答案。 我会在这里试着回答这个问题。

我相信这里的问题是如何设置Apache commons HttpClient（org.apache.commons.httpclient.HttpClient）的代理设置。

下面的代码段应该有效：

 HttpClient client = new HttpClient(); HostConfiguration hostConfiguration = client.getHostConfiguration(); hostConfiguration.setProxy("localhost", 8080); client.setHostConfiguration(hostConfiguration); 

Answer 5

这是我如何使用Santosh Singh（我给了+1）的答案解决了旧的（<4.3）HttpClient（我无法升级）的问题：

 HttpClient httpclient = new HttpClient(); if (System.getProperty("http.proxyHost") != null) { try { HostConfiguration hostConfiguration = httpclient.getHostConfiguration(); hostConfiguration.setProxy(System.getProperty("http.proxyHost"), Integer.parseInt(System.getProperty("http.proxyPort"))); httpclient.setHostConfiguration(hostConfiguration); this.getLogger().warn("USING PROXY: "+httpclient.getHostConfiguration().getProxyHost()); } catch (Exception e) { throw new ProcessingException("Cannot set proxy!", e); } } 

Answer 6

我遇到了与HttpClient版本4类似的问题。

由于SOCKS代理错误，我无法连接到服务器，我使用以下配置修复它：

 client.getParams().setParameter("socksProxyHost",proxyHost); client.getParams().setParameter("socksProxyPort",proxyPort); 

Answer 7

如果您的软件使用ProxySelector （例如使用PAC脚本而不是静态主机/端口）并且您的HTTPComponents是4.3或更高版本，那么您可以将您的ProxySelector用于您的HttpClient如下所示：

 ProxySelector myProxySelector = ...; HttpClient myHttpClient = HttpClientBuilder.create().setRoutePlanner(new SystemDefaultRoutePlanner(myProxySelector))).build(); 

然后像往常一样处理您的请求：

 HttpGet myRequest = new HttpGet("/"); myHttpClient.execute(myRequest);