HTTPS URL的基本代理身份validation返回HTTP / 1.0 407需要代理身份validation
我想在Java中使用具有基本身份validation(用户名,密码)的代理进行连接(并且只有此连接)。 以下代码适用于HTTP URL(例如“ http://www.google.com ”):
URL url = new URL("http://www.google.com"); HttpURLConnection httpURLConnection = null; InetSocketAddress proxyLocation = new InetSocketAddress(proxyHost, proxyPort); Proxy proxy = new Proxy(Proxy.Type.HTTP, proxyLocation); httpURLConnection = (HttpURLConnection) url.openConnection(proxy); // Works for HTTP only! Doesn't work for HTTPS! String encoded = new sun.misc.BASE64Encoder().encodeBuffer((proxyUserName + ":" + proxyPassword).getBytes()).replace("\r\n", ""); httpURLConnection.setRequestProperty("Proxy-Authorization", "Basic " + encoded); InputStream is = httpURLConnection.getInputStream(); InputStreamReader isr = new InputStreamReader(is); int data = isr.read(); while(data != -1){ char c = (char) data; data = isr.read(); System.out.print(c); } isr.close();
但是,该代码不适用于HTTPSurl(例如“ https://www.google.com ”)! 我得到java.io.IOException: Unable to tunnel through proxy. Proxy returns "HTTP/1.0 407 Proxy Authentication Required"
当我尝试访问HTTPS URL时, java.io.IOException: Unable to tunnel through proxy. Proxy returns "HTTP/1.0 407 Proxy Authentication Required"
。
此代码适用于HTTP和HTTPS:
URL url = new URL("https://www.google.com"); HttpURLConnection httpURLConnection = null; InetSocketAddress proxyLocation = new InetSocketAddress(proxyHost, proxyPort); Proxy proxy = new Proxy(Proxy.Type.HTTP, proxyLocation); httpURLConnection = (HttpURLConnection) url.openConnection(proxy); // Works for HTTP and HTTPS, but sets a global default! Authenticator.setDefault(new Authenticator() { protected PasswordAuthentication getPasswordAuthentication() { return new PasswordAuthentication(proxyUserName, proxyPassword.toCharArray()); } }); InputStream is = httpURLConnection.getInputStream(); InputStreamReader isr = new InputStreamReader(is); int data = isr.read(); while(data != -1){ char c = (char) data; data = isr.read(); System.out.print(c); } isr.close();
第二个代码的问题是它设置了一个新的默认Authenticator
,我不想这样做,因为这个代理仅由应用程序的一部分使用,而应用程序的不同部分可能使用不同的代理。 我不想为整个应用程序设置全局默认值。 有没有办法让第一个代码使用HTTPS或使用Authenticator
而不将其设置为默认值?
我必须使用java.net.HttpURLConnection
,因为我重写了一个必须返回HttpURLConnection
的类的方法,所以我不能使用Apache HttpClient。
您可以自己扩展ProxiedHttpsConnection
并处理所有与低级相关的内容。
需要执行以下步骤才能通过HTTP代理连接到https网站:
注意:与代理和http服务器的通信应该是ASCII7 。
- 发送
CONNECT stackoverflow.com:443 HTTP/1.0\r\n
到代理 - 发送您的身份validation:
Proxy-Authorization: Basic c2F5WW91SGF2ZVNlZW5UaGlzSW5UaGVDb21tZW50cw==\r\n
。 - 结束第一个请求:
\r\n
- 从代理中读取响应,直到看到组合“\ r \ n \ r \ n”。
- 解析从代理获得的响应的第一行,并检查它是否以
HTTP/1.0 200
开头。 - 通过现有连接启动SSL会话。
- 发送http请求的开头:
GET /questions/3304006/persistent-httpurlconnection-in-java HTTP/1.0\r\n
- 设置正确的主机头:
Host: stackoverflow.com\r\n
- 结束对http服务器的请求:
\r\n
- 读取
\r\n
并解析第一行作为状态消息 - 读取请求正文的流结束
当我们想要实现HttpUrlConnection类时,我们还需要考虑以下几点:
- 在构造类时,类应该存储未来连接的数据,但不能直接创建
- 可以按任何顺序调用任何方法
-
OutputStream
的关闭意味着数据传输完成,而不是连接必须完成 - 每个api都以不同的顺序使用这些方法
- HTTP标头不区分大小写,java映射区分大小写。
很快就说,有很多陷阱
在我设计的类中,它使用布尔标志来记住是否调用了connect
方法和afterPostClosure
方法,如果在关闭OutputStream
之前调用getInputStream()
,它也有支持。
此类还使用尽可能少的套接字返回的流,以防止真正复杂。
public class ProxiedHttpsConnection extends HttpURLConnection { private final String proxyHost; private final int proxyPort; private static final byte[] NEWLINE = "\r\n".getBytes();//should be "ASCII7" private Socket socket; private final Map> headers = new TreeMap<>(String.CASE_INSENSITIVE_ORDER); private final Map> sendheaders = new TreeMap<>(String.CASE_INSENSITIVE_ORDER); private final Map> proxyheaders = new TreeMap<>(String.CASE_INSENSITIVE_ORDER); private final Map> proxyreturnheaders = new TreeMap<>(String.CASE_INSENSITIVE_ORDER); private int statusCode; private String statusLine; private boolean isDoneWriting; public ProxiedHttpsConnection(URL url, String proxyHost, int proxyPort, String username, String password) throws IOException { super(url); socket = new Socket(); this.proxyHost = proxyHost; this.proxyPort = proxyPort; String encoded = Base64.encode((username + ":" + password).getBytes()) .replace("\r\n", ""); proxyheaders.put("Proxy-Authorization", new ArrayList<>(Arrays.asList("Basic " + encoded))); } @Override public OutputStream getOutputStream() throws IOException { connect(); afterWrite(); return new FilterOutputStream(socket.getOutputStream()) { @Override public void write(byte[] b, int off, int len) throws IOException { out.write(String.valueOf(len).getBytes()); out.write(NEWLINE); out.write(b, off, len); out.write(NEWLINE); } @Override public void write(byte[] b) throws IOException { out.write(String.valueOf(b.length).getBytes()); out.write(NEWLINE); out.write(b); out.write(NEWLINE); } @Override public void write(int b) throws IOException { out.write(String.valueOf(1).getBytes()); out.write(NEWLINE); out.write(b); out.write(NEWLINE); } @Override public void close() throws IOException { afterWrite(); } }; } private boolean afterwritten = false; @Override public InputStream getInputStream() throws IOException { connect(); return socket.getInputStream(); } @Override public void setRequestMethod(String method) throws ProtocolException { this.method = method; } @Override public void setRequestProperty(String key, String value) { sendheaders.put(key, new ArrayList<>(Arrays.asList(value))); } @Override public void addRequestProperty(String key, String value) { sendheaders.computeIfAbsent(key, l -> new ArrayList<>()).add(value); } @Override public Map> getHeaderFields() { return headers; } @Override public void connect() throws IOException { if (connected) { return; } connected = true; socket.setSoTimeout(getReadTimeout()); socket.connect(new InetSocketAddress(proxyHost, proxyPort), getConnectTimeout()); StringBuilder msg = new StringBuilder(); msg.append("CONNECT "); msg.append(url.getHost()); msg.append(':'); msg.append(url.getPort() == -1 ? 443 : url.getPort()); msg.append(" HTTP/1.0\r\n"); for (Map.Entry> header : proxyheaders.entrySet()) { for (String l : header.getValue()) { msg.append(header.getKey()).append(": ").append(l); msg.append("\r\n"); } } msg.append("Connection: close\r\n"); msg.append("\r\n"); byte[] bytes; try { bytes = msg.toString().getBytes("ASCII7"); } catch (UnsupportedEncodingException ignored) { bytes = msg.toString().getBytes(); } socket.getOutputStream().write(bytes); socket.getOutputStream().flush(); byte reply[] = new byte[200]; byte header[] = new byte[200]; int replyLen = 0; int headerLen = 0; int newlinesSeen = 0; boolean headerDone = false; /* Done on first newline */ InputStream in = socket.getInputStream(); while (newlinesSeen < 2) { int i = in.read(); if (i < 0) { throw new IOException("Unexpected EOF from remote server"); } if (i == '\n') { if (newlinesSeen != 0) { String h = new String(header, 0, headerLen); String[] split = h.split(": "); if (split.length != 1) { proxyreturnheaders.computeIfAbsent(split[0], l -> new ArrayList<>()).add(split[1]); } } headerDone = true; ++newlinesSeen; headerLen = 0; } else if (i != '\r') { newlinesSeen = 0; if (!headerDone && replyLen < reply.length) { reply[replyLen++] = (byte) i; } else if (headerLen < reply.length) { header[headerLen++] = (byte) i; } } } String replyStr; try { replyStr = new String(reply, 0, replyLen, "ASCII7"); } catch (UnsupportedEncodingException ignored) { replyStr = new String(reply, 0, replyLen); } // Some proxies return http/1.1, some http/1.0 even we asked for 1.0 if (!replyStr.startsWith("HTTP/1.0 200") && !replyStr.startsWith("HTTP/1.1 200")) { throw new IOException("Unable to tunnel. Proxy returns \"" + replyStr + "\""); } SSLSocket s = (SSLSocket) ((SSLSocketFactory) SSLSocketFactory.getDefault()) .createSocket(socket, url.getHost(), url.getPort(), true); s.startHandshake(); socket = s; msg.setLength(0); msg.append(method); msg.append(" "); msg.append(url.toExternalForm().split(String.valueOf(url.getPort()), -2)[1]); msg.append(" HTTP/1.0\r\n"); for (Map.Entry> h : sendheaders.entrySet()) { for (String l : h.getValue()) { msg.append(h.getKey()).append(": ").append(l); msg.append("\r\n"); } } if (method.equals("POST") || method.equals("PUT")) { msg.append("Transfer-Encoding: Chunked\r\n"); } msg.append("Host: ").append(url.getHost()).append("\r\n"); msg.append("Connection: close\r\n"); msg.append("\r\n"); try { bytes = msg.toString().getBytes("ASCII7"); } catch (UnsupportedEncodingException ignored) { bytes = msg.toString().getBytes(); } socket.getOutputStream().write(bytes); socket.getOutputStream().flush(); } private void afterWrite() throws IOException { if (afterwritten) { return; } afterwritten = true; socket.getOutputStream().write(String.valueOf(0).getBytes()); socket.getOutputStream().write(NEWLINE); socket.getOutputStream().write(NEWLINE); byte reply[] = new byte[200]; byte header[] = new byte[200]; int replyLen = 0; int headerLen = 0; int newlinesSeen = 0; boolean headerDone = false; /* Done on first newline */ InputStream in = socket.getInputStream(); while (newlinesSeen < 2) { int i = in.read(); if (i < 0) { throw new IOException("Unexpected EOF from remote server"); } if (i == '\n') { if (headerDone) { String h = new String(header, 0, headerLen); String[] split = h.split(": "); if (split.length != 1) { headers.computeIfAbsent(split[0], l -> new ArrayList<>()).add(split[1]); } } headerDone = true; ++newlinesSeen; headerLen = 0; } else if (i != '\r') { newlinesSeen = 0; if (!headerDone && replyLen < reply.length) { reply[replyLen++] = (byte) i; } else if (headerLen < header.length) { header[headerLen++] = (byte) i; } } } String replyStr; try { replyStr = new String(reply, 0, replyLen, "ASCII7"); } catch (UnsupportedEncodingException ignored) { replyStr = new String(reply, 0, replyLen); } /* We asked for HTTP/1.0, so we should get that back */ if ((!replyStr.startsWith("HTTP/1.0 200")) && !replyStr.startsWith("HTTP/1.1 200")) { throw new IOException("Server returns \"" + replyStr + "\""); } } @Override public void disconnect() { try { socket.close(); } catch (IOException ex) { Logger.getLogger(ProxiedHttpsConnection.class.getName()).log(Level.SEVERE, null, ex); } } @Override public boolean usingProxy() { return true; } }
目前的错误与上面的代码:
- 在发布期间,流不会因错误而关闭
- 在初始接触代理时出错时,流不会关闭
- 它不支持http重定向
- 它不支持像http和chunked和gzip编码这样的http 1.1,但这不是问题,因为我们宣称自己是一个http1.0客户端。
上面的代码可以像:
ProxiedHttpsConnection n = new ProxiedHttpsConnection( new URL("https://stackoverflow.com:443/questions/3304006/persistent-httpurlconnection-in-java"), "proxy.example.com", 8080, "root", "flg83yvem#"); n.setRequestMethod("GET"); n.addRequestProperty("User-Agent", "Java test https://stackoverflow.com/users/1542723/ferrybig"); //try (OutputStream out = n.getOutputStream()) { // out.write("Hello?".getBytes()); //} try (InputStream in = n.getInputStream()) { byte[] buff = new byte[1024]; int length; while ((length = in.read(buff)) >= 0) { System.out.write(buff, 0, length); } }
如果您要使用某种代理选择器,则应检查URL的协议,以查看其http或https(如果其http)是否不使用此类,而是手动附加标头,如:
httpURLConnection.setRequestProperty("Proxy-Authorization", "Basic " + encoded);
为什么不使用httpsUrlConnection.setSSLSocketFactory
虽然java有这个方法,尝试使用它会告诉你为什么它不起作用,java只是继续用已经打开的连接调用createSocket(Socket s, String host, int port, boolean autoClose)
,这使得它无法做到手动代理的东西。
不幸的是,没有简单的解决方案可以实现您的目标。 您的第一个代码不适用于HTTPS,因为您直接设置了身份validation标头。 由于客户端加密所有数据,因此代理服务器无法从请求中提取任何信息。
事实上,HTTPS和代理服务器以相反的方式工作。 代理服务器希望查看在客户端和最终服务器之间流动的所有数据,并根据其看到的内容采取措施。 另一方面,HTTPS协议对所有数据进行加密,这样在到达最终目的地之前,没有人能够看到数据。 加密算法在客户端和最终目的地之间协商,以便代理服务器不能解密任何信息,实际上它甚至不知道客户端正在使用哪种协议。
要在HTTPS连接上使用代理服务器,客户端必须建立隧道。 为此,它必须直接向代理发出CONNECT命令,例如:
CONNECT www.google.com:443 HTTP/1.0
并发送凭据以通过代理服务器进行身份validation。
如果连接成功,则客户端可以通过连接发送和接收数据。 代理服务器对数据完全失明。 数据仅在客户端和服务器之间的路径上传递。
当您在HTTP URL上执行url.openConnection(proxy)
,它会返回HttpURLConnection
的实例,当在第二个代码中的HTTPS URL上运行时,它会返回HttpsURLConnection
的实例。
您收到407错误代码,因为代理服务器无法从您发送的标头中提取身份validation信息。 查看exception堆栈,我们可以看到sun.net.www.protocol.http.HttpURLConnection.doTunneling()
发出exception,它发出CONNECT命令以通过代理建立HTTPS隧道。 在sun.net.www.protocol.http.HttpURLConnection
的源代码中我们可以看到:
/* We only have a single static authenticator for now. * REMIND: backwards compatibility with JDK 1.1. Should be * eliminated for JDK 2.0. */ private static HttpAuthenticator defaultAuth;
因此,默认身份validation器似乎是提供代理凭据的唯一方法。
要执行您想要的操作,您必须自己进入连接级别并自行处理HTTP协议,因为您必须与代理服务器直接通信才能与Google服务器通信。
你能用HttpsUrlConnection吗? 它扩展了HttpUrlConnection,因此从类返回时,转换为HttpUrlConnection可能没问题。
代码类似,而不是HttpUrlConnection在名称中使用https。
使用以下代码:
if (testUrlHttps.getProtocol().toLowerCase().equals("https")) { trustAllHosts(); HttpsURLConnection https = (HttpsURLConnection) url.openConnection(); https.setHostnameVerifier(DO_NOT_VERYFY); urlCon = https; } else { urlCon = (HttpURLConnection) url.openConnection(); }
资料来源:
[1] https://docs.oracle.com/javase/7/docs/api/javax/net/ssl/HttpsURLConnection.html
[2] HttpURLConnection – “https://”与“http://” (摘录)
好的,这就是你需要做的,
public class ProxyAuth extends Authenticator { private PasswordAuthentication auth; ProxyAuth(String user, String password) { auth = new PasswordAuthentication(user, password == null ? new char[]{} : password.toCharArray()); } protected PasswordAuthentication getPasswordAuthentication() { return auth; } }
。
public class ProxySetup { public HttpURLConnection proxySetup(String urlInput) { URL url; try { url = new URL(urlInput); Proxy proxy = new Proxy(Proxy.Type.HTTP, new InetSocketAddress("10.66.182.100", 80)); // or whatever your proxy is HttpURLConnection uc = (HttpURLConnection)url.openConnection(proxy); System.setProperty("https.proxyHost", "10.66.182.100"); System.setProperty("https.proxyPort", "80"); System.setProperty("http.proxyHost", "10.66.182.100"); System.setProperty("http.proxyPort", "80"); String encoded = new String(Base64.encodeBase64(("domain\\Username" + ":" + "Password").getBytes())); uc.setRequestProperty("Proxy-Authorization", "Basic " + encoded); Authenticator.setDefault(new ProxyAuth("domain\\Username", "Password")); System.out.println("ProxySetup : proxySetup"); return uc; } catch (Exception e) { // TODO Auto-generated catch block System.out.println("ProxySetup : proxySetup - Failed"); e.printStackTrace(); } return null; } }
用它就好。
HttpURLConnection conn = new ProxySetup().proxySetup(URL)
- 如何使用DaoAuthenticationProvider以编程方式使用Spring Security对用户进行身份validation
- 管理身份validation令牌的最佳做法
- 使用JAAS LdapLoginModule通过ActiveDirectory进行身份validation时遇到FailedLoginException
- Spring需要一个’AuthenticationManager’类型的bean
- Spring Security,表单登录和并发会话
- Java – javax.net.ssl.SSLPeerUnverifiedException:peer未经过身份validation
- 如何使JMX自定义身份validation工作?
- spring security从DB获取用户ID
- REST服务上的身份validation令牌是什么意思