HTTP network programming in Java (1): TCP / SSL web page download

Charzueus 2021-02-23 14:34:53
http network programming java tcp


Catalog

One 、 brief introduction :HTTP Programming

    1、HTTP The system design

    2、HTTP Client working process

    3、HTTP The working process of the server

Two 、 be based on TCP Socket Of HTTP Website download

3、 ... and 、 be based on SSL Socket Of HTTPS Website download

Four 、HTTP Client complete code

5、 ... and 、 Interface integrity code

6、 ... and 、 Last + demonstration


One 、 brief introduction :HTTP Programming

Final review HTTP Network programming , Main learning records HTTP(s) Network programming of protocol , Including the use of TCP Socket Three handshakes HTTP Website download , And use SSL Socket Safe transmission of HTTPs Website download , Complete the programming by case practice , know http(s) The actual working mechanism of !

current HTTP The client is much more complex than the earlier ones , It's not just about downloading and displaying web files , There are many new features : Cross platform display 、 Parameter passing 、 The realization of dynamic web page and user interaction, etc .

1、HTTP The system design

  • Client software (web browser :Chrome、360 The browser etc. )
  • Server software (web The server : Microsoft IIS、Apache Tomcat)

2、HTTP Client working process

  • The client software establishes a connection with the server (TCP Three handshakes of );
  • send out HTTP Header format protocol ;
  • Receiving web files ;
  • Show web page .

3、HTTP The working process of the server

  • Turn on the server software 80 port ;
  • Respond to customer requests 、 complete TCP Connect ;
  • Check the client's HTTP Header format to send the web file requested by the customer ( With dynamic web pages ). 

chart 1 HTTP request - Respond to the complete process

Web download technology is a search engine 、 Web crawler 、 Web page collector or network push services and other related applications in the field of basic technology , Here are two protocols used in daily life (http and https) Web access to download .

Two 、 be based on TCP Socket Of HTTP Website download

about TCP Socket connection process has a very deep understanding of , It has also been used to test communication locally TCP Of Socket Establishing a connection , Empathy , And HTTP Server establishes connection , It's also the use of TCP Information exchange .

Once the connection is established , Need to send HTTP Request header , The server confirms the requester , Turn on communication at both ends , The client can receive web file information , After rendering, the web page will be displayed . Here we realize to receive web page file information first , In the next article, we will realize the function of the browser after rendering the web page .

With www.baidu.com For example , And HTTP After the server establishes the connection , We need to send a web request , That is to say HTTP Request header . Construct the request header as follows :

GET / HTTP/1.1  

HOST: www.baidu.com

Accept: */*

Accept-Language: zh-cn

User-Agent: User-Agent:Mozilla/5.0 (Windows NT 10.0; Win64; x64)

Connection: Keep-Alive

  It needs to be sent strictly in accordance with the format , And commonly used StringBuffer Class toString() Method can complete HTTP The request header is converted to a string , Send consistently to HTTP The server .

StringBuffer msg = new StringBuffer();
msg.append("GET / HTTP/1.1\r\n"+
"HOST: "+domainName+"\r\n"+
"Accept: */*\r\n"+
"Accept-Language: zh-CN\r\n"+
"User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64)\r\n"+
"Connection: Keep-Alive\r\n"
);

 

  Line breaks use \r\n To avoid errors due to coding problems .

After sending the request, if the first message returned by the web information display area is “HTTP/1.1 200 OK”, It means that the visit is normal .

You can see HTTP The server returns a lot of information , This is also the response header , Contains a lot of key information content .

3、 ... and 、 be based on SSL Socket Of HTTPS Website download

Based on the above design TCP Communication transmission HTTP, We try to visit www.sina.com.cn, It turns out that the first line of the response header is HTTP/1.1 302 Moved Temporarily( Site removed ), For safety reasons , Now most of the web The site will give up HTTP Enabling HTTPS, They're all using secure encryption HTTPS agreement , And shut it down HTTP, Only enabled SSL/TLS Of HTTPS A secure connection , By default, this connection uses 443 port . therefore TCP Socket There is no normal way to visit the web page .

It's just that the port was changed to 443 Can it be normal , The answer is as follows .

The reason can also be seen in the front , Need to use SSL/TLS Of HTTPS A secure connection , To establish and HTTPS Server communication , So it needs to be modified Socket type .

It's used here Java Secure socket extension (Java Secure Socket Extension,JSSE), be based on SSL and TLS Agreed Java Web applications provide Java API And reference implementation , The client's SSLSocket Socket .SSLSocket Relative to the client socket I learned before , It's just that the creation method is different ,SSLSocket Object by SSLSocketFactory establish .

Declare member variables in a class and create Socket Connect :

private SSLSocket socket;
private SSLSocketFactory factory;
factory=(SSLSocketFactory)SSLSocketFactory.getDefault();
socket=(SSLSocket)factory.createSocket(ip,Integer.parseInt(port));

 

  Yes SSL Socket Use and TCP identical , It's just that the creation method is different , After a little modification , Can successfully request HTTPS Web page information of the website .

Four 、HTTP Client complete code

Here is given HTTP The complete code of the client ,HTTPS Just change the above SSL Socket.

/*
* HTTPClient.java
* Copyright (c) 2020-12-21
* author : Charzous
* All right reserved.
*/
package chapter08;
import java.io.*;
import java.net.Socket;
public class HTTPClient {
private Socket socket;
private PrintWriter pw;
private BufferedReader br;
/**
* @param ip
* @param port
* @return
* @author Charzous
* @date 2020/12/21 14:52
*
*/
public HTTPClient(String ip, String port) throws IOException{
// Actively initiate a connection to the server , Realization TCP Three handshakes
// If you don't succeed, you throw an error , It's up to the caller to handle the error 
socket =new Socket(ip,Integer.parseInt(port));
// Get the network stream output byte stream address , And encapsulate it into network output character stream 
OutputStream socketOut=socket.getOutputStream();
// Parameters true Indicates automatic flush data 
pw=new PrintWriter(new OutputStreamWriter(socketOut,"utf-8"),true);
// Get the network input byte stream address , And encapsulated into a network input character stream 
InputStream socketIn=socket.getInputStream();
br=new BufferedReader(new InputStreamReader(socketIn,"utf-8"));
}
public void send(String msg) throws InterruptedException {
// Output character stream , from socket Call the underlying system function , Send byte stream through network card 
try {
Thread.sleep(500);
}catch (InterruptedException e){
e.printStackTrace();
}
pw.println(msg);
}
public String receive(){
String msg=null;
try {
// Read information from a stream of network input characters , Only one line of information can be accepted at a time
// When you don't have a line ( No line terminator ), The statement blocks
// Until the conditions are met , The program runs down 
msg=br.readLine();
}catch (IOException e){
e.printStackTrace();
}
return msg;
}
public void close(){
try {
if (socket!=null)
socket.close();
}catch (IOException e){
e.printStackTrace();
}
}
}

 

5、 ... and 、 Interface integrity code

I use a graphical interface to access http and https, Integrate the functions of the above two graphics clients , Make the graphic client can access 443 Of https Content , You can also visit non 443 port ( It's usually 80) Of http Content .

/*
* HTTPAllClientFX.java
* Copyright (c) 2020-12-21
* author : Charzous
* All right reserved.
*/
package chapter08;
import javafx.application.Application;
import javafx.application.Platform;
import javafx.geometry.Insets;
import javafx.geometry.Pos;
import javafx.scene.Scene;
import javafx.scene.control.Button;
import javafx.scene.control.Label;
import javafx.scene.control.TextArea;
import javafx.scene.control.TextField;
import javafx.scene.layout.BorderPane;
import javafx.scene.layout.HBox;
import javafx.scene.layout.Priority;
import javafx.scene.layout.VBox;
import javafx.stage.Stage;
public class HTTPAllClientFX extends Application {
private Button btnExit=new Button(" sign out ");
private Button btnSend = new Button(" Web request ");
// private TextField tfSend=new TextField();// Input information area 
private TextArea taDisplay=new TextArea();// Display area 
private TextField ipAddress=new TextField();// Fill in ip Address 
private TextField tfport=new TextField();// Fill in the port 
private Button btConn=new Button(" Connect ");
private HTTPSClient httpsClient;
private HTTPClient httpClient;
private Thread readThread;
public static void main(String[] args) {
launch(args);
}
@Override
public void start(Stage primaryStage) {
BorderPane mainPane=new BorderPane();
// Connect to the server zone 
HBox hBox1=new HBox();
hBox1.setSpacing(10);
hBox1.setPadding(new Insets(10,20,10,20));
hBox1.setAlignment(Pos.CENTER);
hBox1.getChildren().addAll(new Label(" Web address :"),ipAddress,new Label(" port :"),tfport,btConn);
mainPane.setTop(hBox1);
VBox vBox=new VBox();
vBox.setSpacing(10);
vBox.setPadding(new Insets(10,20,10,20));
vBox.getChildren().addAll(new Label(" Web information display area "),taDisplay);
VBox.setVgrow(taDisplay, Priority.ALWAYS);
mainPane.setCenter(vBox);
HBox hBox=new HBox();
hBox.setSpacing(10);
hBox.setPadding(new Insets(10,20,10,20));
hBox.setAlignment(Pos.CENTER_RIGHT);
hBox.getChildren().addAll(btnSend,btnExit);
mainPane.setBottom(hBox);
Scene scene =new Scene(mainPane,700,500);
primaryStage.setScene(scene);
primaryStage.show();
// Connect button 
btConn.setOnAction(event -> {
String ip=ipAddress.getText().trim();
String port=tfport.getText().trim();
taDisplay.clear();
try {
if (port.equals("443")){
httpsClient = new HTTPSClient(ip, port);
// Successfully connected to the server , Accept the first welcome message from the server 
taDisplay.appendText(" Server connection successful .\n");
readThread = new Thread(()->{
String receiveMsg=null;// Receive a string of characters from the server 
if (port.equals("443")){
while ((receiveMsg=httpsClient.receive())!=null){
//lambda You cannot directly access external expressions final Type local variables , You need to define a temporary variable
// If the receiveMsg Defined as a class member variable , You don't need a temporary variable 
String msgTemp = receiveMsg;
Platform.runLater(()->{
taDisplay.appendText(msgTemp+"\n");
});
}
}
});
readThread.start();
}
else if (port.equals("80")){
httpClient = new HTTPClient(ip, port);
taDisplay.appendText(" Server connection successful .\n");
readThread = new Thread(()-> {
String receiveMsg = null;
while ((receiveMsg = httpClient.receive()) != null) {
String msgTemp = receiveMsg;
Platform.runLater(() -> {
taDisplay.appendText(msgTemp + "\n");
});
}
});
readThread.start();
}
}catch (Exception e){
taDisplay.appendText(" Server connection failed !"+e.getMessage()+"\n");
}
});
// Web request button event 
btnSend.setOnAction(event -> {
String ip=ipAddress.getText().trim();
String port=tfport.getText().trim();
String domainName=ipAddress.getText().trim();
try {
StringBuffer msg = new StringBuffer();
msg.append("GET / HTTP/1.1\r\n"+
"HOST: "+domainName+"\r\n"+
"Accept: */*\r\n"+
"Accept-Language: zh-CN\r\n"+
"User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64)\r\n"+
"Connection: Keep-Alive\r\n"
);
if (port.equals("443"))
httpsClient.send(msg.toString());
else if (port.equals("80"))
httpClient.send(msg.toString());
} catch (InterruptedException e) {
e.printStackTrace();
}
});
btnExit.setOnAction(event -> {
try {
exit();
} catch (InterruptedException e) {
e.printStackTrace();
}
});
// Event in response to form closing , Click on the × close , The client is also shut down 
primaryStage.setOnCloseRequest(event -> {
try {
exit();
} catch (InterruptedException e) {
e.printStackTrace();
}
});
}
private void exit() throws InterruptedException {
if (httpsClient!=null||httpClient!=null){
readThread.sleep(1000);// Multithread waiting , There are threads waiting to close the window IO, Set up 1s The interval ensures that all threads are shut down 
 httpsClient.close();
httpClient.close();
}
System.exit(0);
}
}
View Code

6、 ... and 、 Last + demonstration

HTTP Connect www.baidu.com, success

HTTP Connect www.sina.com.cn, Failure

HTTPS Connect www.sina.com.cn, success

Final review , By the way, write a blog , This is the first one , Introduce HTTP Web request download , Mainly HTTP(s) Network programming of protocol , Including the use of TCP Socket Three handshakes HTTP Website download , And use SSL Socket Safe transmission of HTTPs Website download , Complete the programming by case practice , know http(s) The actual working mechanism of !

expect :Java And HTTP Network programming ( The next part : Web browser programming ), Will see the page of HTML Source code , And the web page rendered by browser function !

 

If you think it's good, welcome to “ One key, three links ” Oh , Like, collect, pay attention to , Comment directly if you have questions , Communication and learning !


My blog Park :https://www.cnblogs.com/chenzhenhong/p/14435762.html

my CSDN Blog :https://blog.csdn.net/Charzous/article/details/111470556


 

Copyright notice : This article is an original blog article , follow  CC 4.0 BY-SA  Copyright agreement , For reprint, please attach the original source link and this statement .

 

Link to this article : https://blog.csdn.net/Charzous/article/details/111470556

 

版权声明
本文为[Charzueus]所创,转载请带上原文链接,感谢
https://javamana.com/2021/02/20210223143351452C.html

  1. Add filter and execute filter in excel by Java
  2. Dialogue in spring
  3. 解决Docker MySQL无法被宿主机访问的问题
  4. Oracle OCP 19c 认证1Z0-083考试题库(第1题)
  5. Solve the problem that docker MySQL cannot be accessed by the host
  6. Oracle OCP 19C certification 1z0-083 examination question bank (question 1)
  7. 在 2021 年你需要掌握的 7 种关于 JavaScript 的数组方法
  8. Seven array methods for JavaScript you need to master in 2021
  9. 在 2021 年你需要掌握的 7 种关于 JavaScript 的数组方法
  10. Struts2 + Json _ 配置,异常解决及深入了解Struts2返回JSON数据的原理及具体应用范例
  11. Seven array methods for JavaScript you need to master in 2021
  12. Struts2 + Json _ Configuration, exception resolution and in-depth understanding of Struts2 return JSON data principle and specific application examples
  13. (三)MySQL锁机制 + 事务
  14. (3) MySQL lock mechanism + transaction
  15. 在 2021 年你需要掌握的 7 种关于 JavaScript 的数组方法
  16. Seven array methods for JavaScript you need to master in 2021
  17. 基于Kafka和Elasticsearch构建实时站内搜索功能的实践
  18. Practice of building real time search function in the website based on Kafka and elasticsearch
  19. Golang 实现 Redis(9): 使用GeoHash 搜索附近的人
  20. RxHttp - 轻量级、可扩展、易使用、完美兼容MVVM、MVC架构的网络封装类库
  21. Golang 实现 Redis(9): 使用GeoHash 搜索附近的人
  22. RxHttp - 轻量级、可扩展、易使用、完美兼容MVVM、MVC架构的网络封装类库
  23. Golang realizes redis (9): using geohash to search nearby people
  24. Rxhttp - lightweight, extensible, easy to use, perfectly compatible with MVVM, MVC architecture network encapsulation class library
  25. Golang realizes redis (9): using geohash to search nearby people
  26. Rxhttp - lightweight, extensible, easy to use, perfectly compatible with MVVM, MVC architecture network encapsulation class library
  27. 读懂框架设计的灵魂 — Java 反射机制
  28. 治疗磁盘空间不足焦虑症,释放容器占用空间——Win10+docker篇
  29. 别再用jodatime了!全网最权威Java8日期时间类LocalDate、LocalDateTime详解
  30. Understanding the soul of framework design java reflection mechanism
  31. 配置客户端以安全连接到Apache Kafka集群4:TLS客户端身份验证
  32. Treating anxiety of insufficient disk space and releasing space occupied by containers -- win10 + docker
  33. Don't use jodatime any more! The most authoritative java 8 date and time classes in the whole network: detailed explanation of localdate and localdatetime
  34. Configure clients to connect securely to Apache Kafka Cluster 4: TLS client authentication
  35. Spring break
  36. 高性能MySQL(三):Schema与数据类型优化
  37. High performance mysql (3): schema and data type optimization
  38. redis解决缓存、击穿、雪崩
  39. redis
  40. 骑士卡:基于Kafka搭建消息中心,上亿消息推送轻松完成
  41. Redis solves cache, breakdown and avalanche
  42. redis
  43. Knight card: build a message center based on Kafka, and push hundreds of millions of messages easily
  44. Oracle OCP 19c 认证1Z0-083考试题库(第2题)
  45. redis的三种模式
  46. kubernetes和docker----2.学习Pod资源
  47. 谈一谈如何远程访问MySQL(腾讯云,云主机)
  48. Linux(五):Linux的文档编辑器Vi
  49. Oracle OCP 19C certification 1z0-083 examination question bank (question 2)
  50. 云原生项目实践DevOps(GitOps)+K8S+BPF+SRE,从0到1使用Golang开发生产级麻将游戏服务器—第6篇
  51. kubernetes和docker----2.学习Pod资源
  52. JSP基于Java开发Web应用程序特点有哪些?
  53. Three modes of redis
  54. Kubernetes and docker -- 2. Learning pod resources
  55. Linux (5): the document editor VI of Linux
  56. Cloud native project practice Devops (gitops) + k8s + BPF + SRE, using golang to develop production level mahjong game server from 0 to 1
  57. Kubernetes and docker -- 2. Learning pod resources
  58. What are the characteristics of JSP developing web application based on Java?
  59. Lottie error: java.lang.AssertionError : android.util.JsonReader .peek
  60. Rxhttp - lightweight, extensible, easy to use, perfectly compatible with MVVM, MVC architecture network encapsulation class library