Topic 13: Networking Volume II,Chapter 3 Advanced Programming Techniques
Objective and Outline • Objective: – Introduction to Java networking features • It is much easier to write networking programs inJava than in C++ • But less efficient. • Outline – Motivating example: ICQ Server and Client – Networking basics • IP addresses, ports, protocols, client-server interaction – Socket-level programming • Writing a client (Socket) • Writing a server (ServerSocket) • Example: writing your own icq – Communicating with web servers • Retrieving information (URL, URLConnection) • Sending information
Networking Basics • Internet protocol (IP) addresses – Every host on Internet has a unique IP address 143.89.40.46, 203.184.197.198 203.184.197.196, 203.184.197.197, 127.0.0.1 – More convenient to refer to using hostname string cs.ust.hk, tom.com, localhost – One hostname can correspond to multiple internet addresses: • www.yahoo.com: 66.218.70.49; 66.218.70.50; 66.218.71.80; 66.218.71.84; … – Domain Naming Service (DNS) maps names to numbers
 java.net.InetAddress class converts between hostnames and internet addresses InetAddress tm = InetAddress.getByName(“www.yahoo.com"); InetAddress tm= InetAddress.getByName(“localhost"); //127.0.0.1 InetAddress tm = InetAddress.getLocalHost();  Can get array of addresses (if more than one) InetAddress[] addrs; addrs=InetAddress.getAllByName(“www.yahoo.com"); for (int i = 0; i < addr.length; i++) System.out.println(addrs[i].getHostAddress()); InetAddressTest.java Networking Basics
 Ports  Many different services can be running on the host  A port identifies a service within a host  Many standard port numbers are pre-assigned time of day 13, ftp 21, telnet 23, smtp 25, http 80 see /etc/services on workstation for list of all assigned ports  IP address + port number = "phone number“ for service Networking Basics
 protocols : rules that facilitate communications between machines  Examples:  HTTP: HyperText Transfer Protocol  FTP: File Transfer Protocol  SMTP: Simple Message Transfer Protocol  TCP: Transmission Control Protocol  UDP: User Datagram Protocol, good for, e.g., video delivery)  Protocols are standardized and documented So machines can reliably work with one another Networking Basics
 Client-Server interaction  Communication between hosts is two-way, but usually the two hosts take different roles  Server waits for client to make request Server registered on a known port with the host ("public phone number") Usually running in endless loop Listens for incoming client connections Networking Basics
 Client "calls" server to start a conversation Client making calls uses hostname/IP address and port number Sends request and waits for response  Standard services always running ftp, http, smtp, etc. server running on host using expected port  Server offers shared resource (information,database, files, printer, compute power) to clients Networking Basics
 Using telnet to try out some services of servers:  Telnet assumes you want to connect to port 23 on the receiving host (port 23 is where the telnet server is listening)  However there is an optional argument after the hostname that allows you to connect to a different port  Try the following Get time: telnet time-A.timefreq.bldrdoc.gov 13 Get HTML page: telnet www.cs.ust.hk 80 and enter a GET command  Many servers now refuse telnet connections due to security reasons. Networking Basics
Outline • Outline – Networking basics • IP addresses, ports, protocols, client-server interaction – Socket-level programming • Writing a client • Writing a server • Example: writing your own icq – Communicating with web servers • Retrieving information • Sending information
Socket-Level Programming  Socket is an abstraction of one type of bi-directional communication channel between hosts  Send and receive data using streams  Next:  How to write a client  How to write a server Client Server OutputStream InputStream InputStream OutputStream
• To write a client socket using java.net.Socket – Create a new Socket with hostname and port number of the connection Socket s = New Socket(String hostName,int portNumber); – Call s.getOutputStream() and s.getInputStream() to get streams for sending and receiving infomation – Need to learn protocol used to communicate • Know how to properly form requests to send to server • Know how to interpret the server’s responses Writing Clients
Writing Clients  SocketTest: Makes a socket connection to the atomic clock in Boulder, Colorado, and prints the time that the server sends. try { Socket s = new Socket("time-A.timefreq.bldrdoc.gov", 13); BufferedReader in = new BufferedReader (new InputStreamReader( s.getInputStream() )); // read from in } catch (IOException e) { e.printStackTrace(); }
Writing Servers  To write a server using java.net.ServerSocket  Create a new ServerSocket with a port number to listen on the port ServerSocket s = New ServerSocket( portNumber);  Use accept() to listen on the port.  accept() returns a socket incoming when a client calls Socket incoming = s.accept();  Call incoming.getOutputStream() and incoming.getInputStream() to get streams for sending and receiving information
Writing Servers  Example: Echo server ServerSocket s = new ServerSocket(8189); Socket incoming = s.accept( ); BufferedReader in = new BufferedReader (new InputStreamReader(incoming.getInputStream())); PrintWriter out = new PrintWriter (incoming.getOutputStream(), true /* autoFlush */ ); out.println( "Hello! Enter BYE to exit." ); … EchoServer.java
A side note • Many machines in CSD now refuse socket connections due to security considerations. • However, you can – Run severs on any lab 4 machine and connect to the server from any other lab 4 machines. – Run severs on one of scpu1-14 and connect to the server from others or from the PC network.
 Multithread server: starts a separate thread for each connection. public class ThreadedEchoServer { public static void main(String[] args ) { int i = 1; try{ServerSocket s = new ServerSocket(8190); while (true) { Socket incoming = s.accept( ); System.out.println("Spawning " + i); new ThreadedEchoHandler(incoming, i).start(); i++; } } catch (Exception e) …. //ThreadedEchoServer.java Writing Servers
class ThreadedEchoHandler extends Thread { public ThreadedEchoHandler(Socket i, int c) { incoming = i; counter = c; } public void run() { try { BufferedReader in = new BufferedReader (new InputStreamReader(incoming.getInputStream())); PrintWriter out = new PrintWriter (incoming.getOutputStream(), true /* autoFlush */); out.println( "Hello! Enter BYE to exit." ); … private Socket incoming; private int counter; } Writing Servers
• A more interesting example – ICQServer.java • A simple server that listens on port 7777. • Connect two clients so that they can talk to each other. • Can handle more than one pairs. – ICQClient.java • Allows user to connect to ICQServer and have one-to- one conversation with partner Servers & Client
Outline • Outline – Networking basics • IP addresses, ports, protocols, client-server interaction – Socket-level programming • Writing a client • Writing a server • Example: writing your own icq – Communicating with web servers • Retrieving information • Sending information
• Reason for communicating with web servers – To retrieve/send information • Need to indicate location of resource – URL stands for Uniform Resource Locator • Neat scheme for uniquely identifying all kinds of network resources – Basic form <protocol>:<sitename><pathname> • http://www.cs.ust.hk/~lzhang/comp201/index.html • ftp://ftp.cs.ust.hk/pub/lzhang/teach/201/codes/HttpTest/HttpTest.jav a • file:/MyDisk/Letters/ToMom2-11-98 • Protocols include files, http, ftp, gopher, news, mailto, etc. Communicating with web servers
• Class java.net.URL represents a Uniform Resource Locator – Create an java object that represents an URL URL url = new URL(“http://www.cs.ust.hk/~lzhang/comp201/index.html”); • getHost(), getPath(), getPort(), getProtocol() • java.net.URLConnection represents a communication link between the application and a URL. – Constructor: • URLConnection cnn = new URLConnection( url) – Obtainable also from URL: • URLConnection cnn = url.openConnection(); Communicating with web servers
Communicating with web servers • Steps for working with java.net.URLConnection – Set properties of connection: • setDoInPut(true) //default • setDoOutPut(true) for sending information to the server • … – Make connection: cnn.connect(); – Query header information: • getContentType, getContentLength, getContentEncoding, getDate, getExpiration, getLastModified – getInputStream for reading and getOutputStream for writing • API of the class has a more detailed description.
Communicating with web servers • Can directly open a stream for reading in URL class: – public final InputStream openStream() throws IOException url.opentStream() • Opens a connection to this URL and returns an InputStream for reading from that connection. • This method is a shorthand for: openConnection().getInputStream() URLTest.java
Retrieving Information • URLConnectionTest.java URL url = new URL(urlName); URLConnection connection = url.openConnection(); connection.connect(); // print header fields int n = 1; String key; while ((key = connection.getHeaderFieldKey(n)) != null) { String value = connection.getHeaderField(n); System.out.println(key + ": " + value); n++; }
Retrieving Information // print convenience functions System.out.println("----------"); System.out.println("getContentType: " + connection.getContentType() ); System.out.println("getContentLength: " + connection.getContentLength() ); System.out.println("getContentEncoding: " + connection.getContentEncoding() ); ….
Retrieving Information // print first ten lines of contents BufferedReader in = new BufferedReader(new InputStreamReader( connection.getInputStream() )); String line; n = 1; while ((line = in.readLine()) != null && n <= 10) { System.out.println(line); n++; } if (line != null) System.out.println(". . .");
Sending Information  Web servers receive information from clients using either GET or POST  GET requests are requests made by browsers when the user  types in a URL on the address line,  follows a link from a Web page, or  makes an HTML form that does not specify a METHOD or specifically use the GET method.  POST requests are generated when someone creates an HTML form that specifies METHOD="POST"  Examples:  http://maps.yahoo.com/py/maps.py: python, <form action="/py/maps.py?Pyt=Tmap&YY=28457" method=GET> … </form>  http://www.census.gov/ipc/www/idbprint.html: <form method=post action="/cgi-bin/ipc/idbsprd">
Sending Information  Appropriate CGI (common gateway interface) script is called to process info received and produce an HTML page to send back to client  CGI scripts usually written in C, Perl, shell script. (Out of the scope of this course.)  Will discuss servlets, Java alternative to CGI scripts
• Our task: Write java program to communicate with CGI scripts – The way we send parameters to a CGI script depends on • The parameters that CGI scripts expects – What to send • The way a CGI script receives parameters – How to send Sending Information
 Send information to CGI script using GET  Attach parameters to the end of URL http://host/script?parameters  Separate parameters using “&” and encode parameters as follows to avoid misinterpretation (URL encoding)  Replace space with “+”  Replace each non-alphanumeric character with “%” followed by the hexadecimal code of the character “Mastering C++”  “Mastering+C%2b%2b”  Disadvantage: long parameter string, might exceed limits of browsers. GetTest.java Sending Information
 Sending information to CGI script using POST: Open URLConnection and send parameter using a stream  Open a URLConnection: URL url = new URL(“http:/host/script”); URLConnection cnn = url.openConnection();  Set up connection for output: cnn.setDoOutput(true); Sending Information
 Get a stream for sending data: PrinterWriter out = new PrintWriter(cnn.getOutputStream());  Send parameters Out.print(name1 + “=“ + URLEncoder.encode(value1, “UTF-8”) + “&” ); Out.print(name2 + “=“ + URLEncoder.encode(value2, “UTF-8”) ) + “n”); Note: URLEncoder: Utility class for HTML form encoding. This class contains static methods for converting a String to the application/x-www- form-urlencoded MIME (Multipurpose Internet Mail Extensions ) format. The World Wide Web Consortium Recommendation states that the UTF-8 encoding scheme should be used. PostTest.java Sending Information

java networking

  • 1.
    Topic 13: Networking VolumeII,Chapter 3 Advanced Programming Techniques
  • 2.
    Objective and Outline •Objective: – Introduction to Java networking features • It is much easier to write networking programs inJava than in C++ • But less efficient. • Outline – Motivating example: ICQ Server and Client – Networking basics • IP addresses, ports, protocols, client-server interaction – Socket-level programming • Writing a client (Socket) • Writing a server (ServerSocket) • Example: writing your own icq – Communicating with web servers • Retrieving information (URL, URLConnection) • Sending information
  • 3.
    Networking Basics • Internetprotocol (IP) addresses – Every host on Internet has a unique IP address 143.89.40.46, 203.184.197.198 203.184.197.196, 203.184.197.197, 127.0.0.1 – More convenient to refer to using hostname string cs.ust.hk, tom.com, localhost – One hostname can correspond to multiple internet addresses: • www.yahoo.com: 66.218.70.49; 66.218.70.50; 66.218.71.80; 66.218.71.84; … – Domain Naming Service (DNS) maps names to numbers
  • 4.
     java.net.InetAddress classconverts between hostnames and internet addresses InetAddress tm = InetAddress.getByName(“www.yahoo.com"); InetAddress tm= InetAddress.getByName(“localhost"); //127.0.0.1 InetAddress tm = InetAddress.getLocalHost();  Can get array of addresses (if more than one) InetAddress[] addrs; addrs=InetAddress.getAllByName(“www.yahoo.com"); for (int i = 0; i < addr.length; i++) System.out.println(addrs[i].getHostAddress()); InetAddressTest.java Networking Basics
  • 5.
     Ports  Manydifferent services can be running on the host  A port identifies a service within a host  Many standard port numbers are pre-assigned time of day 13, ftp 21, telnet 23, smtp 25, http 80 see /etc/services on workstation for list of all assigned ports  IP address + port number = "phone number“ for service Networking Basics
  • 6.
     protocols :rules that facilitate communications between machines  Examples:  HTTP: HyperText Transfer Protocol  FTP: File Transfer Protocol  SMTP: Simple Message Transfer Protocol  TCP: Transmission Control Protocol  UDP: User Datagram Protocol, good for, e.g., video delivery)  Protocols are standardized and documented So machines can reliably work with one another Networking Basics
  • 7.
     Client-Server interaction Communication between hosts is two-way, but usually the two hosts take different roles  Server waits for client to make request Server registered on a known port with the host ("public phone number") Usually running in endless loop Listens for incoming client connections Networking Basics
  • 8.
     Client "calls"server to start a conversation Client making calls uses hostname/IP address and port number Sends request and waits for response  Standard services always running ftp, http, smtp, etc. server running on host using expected port  Server offers shared resource (information,database, files, printer, compute power) to clients Networking Basics
  • 9.
     Using telnetto try out some services of servers:  Telnet assumes you want to connect to port 23 on the receiving host (port 23 is where the telnet server is listening)  However there is an optional argument after the hostname that allows you to connect to a different port  Try the following Get time: telnet time-A.timefreq.bldrdoc.gov 13 Get HTML page: telnet www.cs.ust.hk 80 and enter a GET command  Many servers now refuse telnet connections due to security reasons. Networking Basics
  • 10.
    Outline • Outline – Networkingbasics • IP addresses, ports, protocols, client-server interaction – Socket-level programming • Writing a client • Writing a server • Example: writing your own icq – Communicating with web servers • Retrieving information • Sending information
  • 11.
    Socket-Level Programming  Socketis an abstraction of one type of bi-directional communication channel between hosts  Send and receive data using streams  Next:  How to write a client  How to write a server Client Server OutputStream InputStream InputStream OutputStream
  • 12.
    • To writea client socket using java.net.Socket – Create a new Socket with hostname and port number of the connection Socket s = New Socket(String hostName,int portNumber); – Call s.getOutputStream() and s.getInputStream() to get streams for sending and receiving infomation – Need to learn protocol used to communicate • Know how to properly form requests to send to server • Know how to interpret the server’s responses Writing Clients
  • 13.
    Writing Clients  SocketTest: Makesa socket connection to the atomic clock in Boulder, Colorado, and prints the time that the server sends. try { Socket s = new Socket("time-A.timefreq.bldrdoc.gov", 13); BufferedReader in = new BufferedReader (new InputStreamReader( s.getInputStream() )); // read from in } catch (IOException e) { e.printStackTrace(); }
  • 14.
    Writing Servers  Towrite a server using java.net.ServerSocket  Create a new ServerSocket with a port number to listen on the port ServerSocket s = New ServerSocket( portNumber);  Use accept() to listen on the port.  accept() returns a socket incoming when a client calls Socket incoming = s.accept();  Call incoming.getOutputStream() and incoming.getInputStream() to get streams for sending and receiving information
  • 15.
    Writing Servers  Example:Echo server ServerSocket s = new ServerSocket(8189); Socket incoming = s.accept( ); BufferedReader in = new BufferedReader (new InputStreamReader(incoming.getInputStream())); PrintWriter out = new PrintWriter (incoming.getOutputStream(), true /* autoFlush */ ); out.println( "Hello! Enter BYE to exit." ); … EchoServer.java
  • 16.
    A side note •Many machines in CSD now refuse socket connections due to security considerations. • However, you can – Run severs on any lab 4 machine and connect to the server from any other lab 4 machines. – Run severs on one of scpu1-14 and connect to the server from others or from the PC network.
  • 17.
     Multithread server:starts a separate thread for each connection. public class ThreadedEchoServer { public static void main(String[] args ) { int i = 1; try{ServerSocket s = new ServerSocket(8190); while (true) { Socket incoming = s.accept( ); System.out.println("Spawning " + i); new ThreadedEchoHandler(incoming, i).start(); i++; } } catch (Exception e) …. //ThreadedEchoServer.java Writing Servers
  • 18.
    class ThreadedEchoHandler extendsThread { public ThreadedEchoHandler(Socket i, int c) { incoming = i; counter = c; } public void run() { try { BufferedReader in = new BufferedReader (new InputStreamReader(incoming.getInputStream())); PrintWriter out = new PrintWriter (incoming.getOutputStream(), true /* autoFlush */); out.println( "Hello! Enter BYE to exit." ); … private Socket incoming; private int counter; } Writing Servers
  • 19.
    • A moreinteresting example – ICQServer.java • A simple server that listens on port 7777. • Connect two clients so that they can talk to each other. • Can handle more than one pairs. – ICQClient.java • Allows user to connect to ICQServer and have one-to- one conversation with partner Servers & Client
  • 20.
    Outline • Outline – Networkingbasics • IP addresses, ports, protocols, client-server interaction – Socket-level programming • Writing a client • Writing a server • Example: writing your own icq – Communicating with web servers • Retrieving information • Sending information
  • 21.
    • Reason forcommunicating with web servers – To retrieve/send information • Need to indicate location of resource – URL stands for Uniform Resource Locator • Neat scheme for uniquely identifying all kinds of network resources – Basic form <protocol>:<sitename><pathname> • http://www.cs.ust.hk/~lzhang/comp201/index.html • ftp://ftp.cs.ust.hk/pub/lzhang/teach/201/codes/HttpTest/HttpTest.jav a • file:/MyDisk/Letters/ToMom2-11-98 • Protocols include files, http, ftp, gopher, news, mailto, etc. Communicating with web servers
  • 22.
    • Class java.net.URLrepresents a Uniform Resource Locator – Create an java object that represents an URL URL url = new URL(“http://www.cs.ust.hk/~lzhang/comp201/index.html”); • getHost(), getPath(), getPort(), getProtocol() • java.net.URLConnection represents a communication link between the application and a URL. – Constructor: • URLConnection cnn = new URLConnection( url) – Obtainable also from URL: • URLConnection cnn = url.openConnection(); Communicating with web servers
  • 23.
    Communicating with webservers • Steps for working with java.net.URLConnection – Set properties of connection: • setDoInPut(true) //default • setDoOutPut(true) for sending information to the server • … – Make connection: cnn.connect(); – Query header information: • getContentType, getContentLength, getContentEncoding, getDate, getExpiration, getLastModified – getInputStream for reading and getOutputStream for writing • API of the class has a more detailed description.
  • 24.
    Communicating with webservers • Can directly open a stream for reading in URL class: – public final InputStream openStream() throws IOException url.opentStream() • Opens a connection to this URL and returns an InputStream for reading from that connection. • This method is a shorthand for: openConnection().getInputStream() URLTest.java
  • 25.
    Retrieving Information • URLConnectionTest.java URLurl = new URL(urlName); URLConnection connection = url.openConnection(); connection.connect(); // print header fields int n = 1; String key; while ((key = connection.getHeaderFieldKey(n)) != null) { String value = connection.getHeaderField(n); System.out.println(key + ": " + value); n++; }
  • 26.
    Retrieving Information // printconvenience functions System.out.println("----------"); System.out.println("getContentType: " + connection.getContentType() ); System.out.println("getContentLength: " + connection.getContentLength() ); System.out.println("getContentEncoding: " + connection.getContentEncoding() ); ….
  • 27.
    Retrieving Information // printfirst ten lines of contents BufferedReader in = new BufferedReader(new InputStreamReader( connection.getInputStream() )); String line; n = 1; while ((line = in.readLine()) != null && n <= 10) { System.out.println(line); n++; } if (line != null) System.out.println(". . .");
  • 28.
    Sending Information  Webservers receive information from clients using either GET or POST  GET requests are requests made by browsers when the user  types in a URL on the address line,  follows a link from a Web page, or  makes an HTML form that does not specify a METHOD or specifically use the GET method.  POST requests are generated when someone creates an HTML form that specifies METHOD="POST"  Examples:  http://maps.yahoo.com/py/maps.py: python, <form action="/py/maps.py?Pyt=Tmap&YY=28457" method=GET> … </form>  http://www.census.gov/ipc/www/idbprint.html: <form method=post action="/cgi-bin/ipc/idbsprd">
  • 29.
    Sending Information  AppropriateCGI (common gateway interface) script is called to process info received and produce an HTML page to send back to client  CGI scripts usually written in C, Perl, shell script. (Out of the scope of this course.)  Will discuss servlets, Java alternative to CGI scripts
  • 30.
    • Our task:Write java program to communicate with CGI scripts – The way we send parameters to a CGI script depends on • The parameters that CGI scripts expects – What to send • The way a CGI script receives parameters – How to send Sending Information
  • 31.
     Send informationto CGI script using GET  Attach parameters to the end of URL http://host/script?parameters  Separate parameters using “&” and encode parameters as follows to avoid misinterpretation (URL encoding)  Replace space with “+”  Replace each non-alphanumeric character with “%” followed by the hexadecimal code of the character “Mastering C++”  “Mastering+C%2b%2b”  Disadvantage: long parameter string, might exceed limits of browsers. GetTest.java Sending Information
  • 32.
     Sending informationto CGI script using POST: Open URLConnection and send parameter using a stream  Open a URLConnection: URL url = new URL(“http:/host/script”); URLConnection cnn = url.openConnection();  Set up connection for output: cnn.setDoOutput(true); Sending Information
  • 33.
     Get astream for sending data: PrinterWriter out = new PrintWriter(cnn.getOutputStream());  Send parameters Out.print(name1 + “=“ + URLEncoder.encode(value1, “UTF-8”) + “&” ); Out.print(name2 + “=“ + URLEncoder.encode(value2, “UTF-8”) ) + “n”); Note: URLEncoder: Utility class for HTML form encoding. This class contains static methods for converting a String to the application/x-www- form-urlencoded MIME (Multipurpose Internet Mail Extensions ) format. The World Wide Web Consortium Recommendation states that the UTF-8 encoding scheme should be used. PostTest.java Sending Information