Project

General

Profile

Actions

Feature #4142

closed

multipart/form-data for net/http

Feature #4142: multipart/form-data for net/http

Added by naruse (Yui NARUSE) almost 15 years ago. Updated over 14 years ago.

Status:
Closed
Assignee:
-
Target version:
-
[ruby-dev:42729]

Description

=begin
multipart/form-data 対応を net/http に入れませんか。
追加される API は Net::HTTPRequest#set_form になります。

akr さんからは multipart/form-data 用のデータを出力する API 案も示唆されたのですが、
chunked encoding を考慮に入れるとうまくまとまらなかったので見送っています。

diff --git a/lib/net/http.rb b/lib/net/http.rb
index 4d475b1..2751f77 100644
--- a/lib/net/http.rb
+++ b/lib/net/http.rb
@@ -22,6 +22,7 @@
require 'net/protocol'
autoload :OpenSSL, 'openssl'
require 'uri'
+autoload :SecureRandom, 'securerandom'

module Net #:nodoc:

@@ -1772,7 +1773,8 @@ module Net #:nodoc:
alias content_type= set_content_type

 # Set header fields and a body from HTML form data. 
  • +params+ should be a Hash containing HTML form data.

  • +params+ should be an Array of Arrays or

  • a Hash containing HTML form data.

    Optional argument +sep+ means data record separator.

    Values are URL encoded as necessary and the content-type is set to

@@ -1792,6 +1794,48 @@ module Net #:nodoc:

 alias form_data= set_form_data 
  • Set a HTML form data set.

  • +params+ is the form data set; it is an Array of Arrays or a Hash

  • +enctype is the type to encode the form data set.

  • It is application/x-www-form-urlencoded or multipart/form-data.

  • +formpot+ is an optional hash to specify the detail.

  • boundary:: the boundary of the multipart message

  • charset:: the charset of the message. All names and the values of

  • non-file fields are encoded as the charset.

  • Each item of params is an array and contains following items:

  • +name+:: the name of the field

  • +value+:: the value of the field, it should be a String or a File

  • +opt+:: an optional hash to specify additional information

  • Each item is a file field or a normal field.

  • If +value+ is a File object or the +opt+ have a filename key,

  • the item is treated as a file field.

  • If Transfer-Encoding is set as chunked, this send the request in

  • chunked encoding. Because chunked encoding is HTTP/1.1 feature,

  • you must confirm the server to support HTTP/1.1 before sending it.

  • Example:

  • http.set_form([["q", "ruby"], ["lang", "en"]])

  • See also RFC 2388, RFC 2616, HTML 4.01, and HTML5

  • def set_form(params, enctype='application/x-www-form-urlencoded', formopt={})

  •  @body_data = params 
  •  @body = nil 
  •  @body_stream = nil 
  •  @form_option = formopt 
  •  case enctype 
  •  when /\Aapplication\/x-www-form-urlencoded\z/i, 
  •  /\Amultipart\/form-data\z/i 
  •  self.content_type = enctype 
  •  else 
  •  raise ArgumentError, "invalid enctype: #{enctype}" 
  •  end 
  • end

  • Set the Authorization: header for "Basic" authorization.

    def basic_auth(account, password)
    @header['authorization'] = [basic_encode(account, password)]
    @@ -1849,6 +1893,7 @@ module Net #:nodoc:
    self['User-Agent'] ||= 'Ruby'
    @body = nil
    @body_stream = nil

  •  @body_data = nil 

    end

    attr_reader :method
    @@ -1876,6 +1921,7 @@ module Net #:nodoc:
    def body=(str)
    @body = str
    @body_stream = nil

  •  @body_data = nil str 

    end

@@ -1884,6 +1930,7 @@ module Net #:nodoc:
def body_stream=(input)
@body = nil
@body_stream = input

  •  @body_data = nil input 
    end

@@ -1901,6 +1948,8 @@ module Net #:nodoc:
send_request_with_body sock, ver, path, @body
elsif @body_stream
send_request_with_body_stream sock, ver, path, @body_stream

  •  elsif @body_data 
  •  send_request_with_body_data sock, ver, path, @body_data else write_header sock, ver, path end 

@@ -1935,6 +1984,92 @@ module Net #:nodoc:
end
end

  • def send_request_with_body_data(sock, ver, path, params)

  •  if /\Amultipart\/form-data\z/i !~ self.content_type 
  •  self.content_type = 'application/x-www-form-urlencoded' 
  •  return send_request_with_body(sock, ver, path, URI.encode_www_form(params)) 
  •  end 
  •  opt = @form_option.dup 
  •  opt[:boundary] ||= SecureRandom.urlsafe_base64(40) 
  •  self.set_content_type(self.content_type, boundary: opt[:boundary]) 
  •  if chunked? 
  •  write_header sock, ver, path 
  •  encode_multipart_form_data(sock, params, opt) 
  •  else 
  •  require 'tempfile' 
  •  file = Tempfile.new('multipart') 
  •  encode_multipart_form_data(file, params, opt) 
  •  file.rewind 
  •  self.content_length = file.size 
  •  write_header sock, ver, path 
  •  IO.copy_stream(file, sock) 
  •  end 
  • end

  • def encode_multipart_form_data(out, params, opt)

  •  charset = opt[:charset] 
  •  boundary = opt[:boundary] 
  •  boundary ||= SecureRandom.urlsafe_base64(40) 
  •  chunked_p = chunked? 
  •  buf = '' 
  •  params.each do |key, value, h={}| 
  •  key = quote_string(key, charset) 
  •  filename = 
  •  h.key?(:filename) ? h[:filename] : 
  •  value.respond_to?(:to_path) ? File.basename(value.to_path) : 
  •  nil 
  •  buf << "--#{boundary}\r\n" 
  •  if filename 
  •  filename = quote_string(filename, charset) 
  •  type = h[:content_type] || 'application/octet-stream' 
  •  buf << "Content-Disposition: form-data; " \ 
  •  "name=\"#{key}\"; filename=\"#{filename}\"\r\n" \ 
  •  "Content-Type: #{type}\r\n\r\n" 
  •  if !out.respond_to?(:write) || !value.respond_to?(:read) 
  •  # if +out+ is not an IO or +value+ is not an IO 
  •  buf << (value.respond_to?(:read) ? value.read : value) 
  •  elsif value.respond_to?(:size) && chunked_p 
  •  # if +out+ is an IO and +value+ is a File, use IO.copy_stream 
  •  flush_buffer(out, buf, chunked_p) 
  •  out << "%x\r\n" % value.size if chunked_p 
  •  IO.copy_stream(value, out) 
  •  out << "\r\n" if chunked_p 
  •  else 
  •  # +out+ is an IO, and +value+ is not a File but an IO 
  •  flush_buffer(out, buf, chunked_p) 
  •  1 while flush_buffer(out, value.read(4096), chunked_p) 
  •  end 
  •  else 
  •  # non-file field: 
  •  # HTML5 says, "The parts of the generated multipart/form-data 
  •  # resource that correspond to non-file fields must not have a 
  •  # Content-Type header specified." 
  •  buf << "Content-Disposition: form-data; name=\"#{key}\"\r\n\r\n" 
  •  buf << (value.respond_to?(:read) ? value.read : value) 
  •  end 
  •  buf << "\r\n" 
  •  end 
  •  buf << "--#{boundary}--\r\n" 
  •  flush_buffer(out, buf, chunked_p) 
  •  out << "0\r\n\r\n" if chunked_p 
  • end

  • def quote_string(str, charset)

  •  str = str.encode(charset, fallback:->(c){'&#%d;'%c.encode("UTF-8").ord}) if charset 
  •  str = str.gsub(/[\\"]/, '\\\\\&') 
  • end

  • def flush_buffer(out, buf, chunked_p)

  •  return unless buf 
  •  out << "%x\r\n"%buf.bytesize if chunked_p 
  •  out << buf 
  •  out << "\r\n" if chunked_p 
  •  buf.clear 
  • end

  • def supply_default_content_type
    return if content_type()
    warn 'net/http: warning: Content-Type did not set; using application/x-www-form-urlencoded' if $VERBOSE
    diff --git a/lib/net/protocol.rb b/lib/net/protocol.rb
    index 2a6cfb4..a3ffa71 100644
    --- a/lib/net/protocol.rb
    +++ b/lib/net/protocol.rb
    @@ -168,6 +168,8 @@ module Net # :nodoc:
    }
    end

  • alias << write

  • def writeline(str)
    writing {
    write0 str + "\r\n"
    diff --git a/test/net/http/test_http.rb b/test/net/http/test_http.rb
    index 76280ad..12c03a4 100644
    --- a/test/net/http/test_http.rb
    +++ b/test/net/http/test_http.rb
    @@ -303,6 +303,102 @@ module TestNetHTTP_version_1_2_methods
    assert_equal data.size, res.body.size
    assert_equal data, res.body
    end

  • def test_set_form

  • require 'tempfile'

  • file = Tempfile.new('ruby-test')

  • file << "\u{30c7}\u{30fc}\u{30bf}"

  • data = [

  •  ['name', 'Gonbei Nanashi'], 
  •  ['name', "\u{540d}\u{7121}\u{3057}\u{306e}\u{6a29}\u{5175}\u{885b}"], 
  •  ['s"i\o', StringIO.new("\u{3042 3044 4e9c 925b}")], 
  •  ["file", file, filename: "ruby-test"] 
  • ]

  • expected = <<"EOM".gsub(/\n/, "\r\n")
    +--
    +Content-Disposition: form-data; name="name"

+Gonbei Nanashi
+--
+Content-Disposition: form-data; name="name"
+
+\xE5\x90\x8D\xE7\x84\xA1\xE3\x81\x97\xE3\x81\xAE\xE6\xA8\xA9\xE5\x85\xB5\xE8\xA1\x9B
+--
+Content-Disposition: form-data; name="s\"i\\o"
+
+\xE3\x81\x82\xE3\x81\x84\xE4\xBA\x9C\xE9\x89\x9B
+--
+Content-Disposition: form-data; name="file"; filename="ruby-test"
+Content-Type: application/octet-stream
+
+\xE3\x83\x87\xE3\x83\xBC\xE3\x82\xBF
+----
+EOM

  • start {|http|
  •  _test_set_form_urlencoded(http, data.reject{|k,v|!v.is_a?(String)}) 
  •  _test_set_form_multipart(http, false, data, expected) 
  •  _test_set_form_multipart(http, true, data, expected) 
  • }
  • end
  • def _test_set_form_urlencoded(http, data)
  • req = Net::HTTP::Post.new('/')
  • req.set_form(data)
  • res = http.request req
  • assert_equal "name=Gonbei+Nanashi&name=%E5%90%8D%E7%84%A1%E3%81%97%E3%81%AE%E6%A8%A9%E5%85%B5%E8%A1%9B", res.body
  • end
  • def _test_set_form_multipart(http, chunked_p, data, expected)
  • data.each{|k,v|v.rewind rescue nil}
  • req = Net::HTTP::Post.new('/')
  • req.set_form(data, 'multipart/form-data')
  • req['Transfer-Encoding'] = 'chunked' if chunked_p
  • res = http.request req
  • body = res.body
  • assert_match(/\A--(?\S+)/, body)
  • /\A--(?\S+)/ =~ body
  • expected = expected.gsub(//, boundary)
  • assert_equal(expected, body)
  • end
  • def test_set_form_with_file
  • require 'tempfile'
  • file = Tempfile.new('ruby-test')
  • file << $test_net_http_data
  • filename = File.basename(file.to_path)
  • data = [['file', file]]
  • expected = <<"EOM".gsub(/\n/, "\r\n")
    +--
    +Content-Disposition: form-data; name="file"; filename=""
    +Content-Type: application/octet-stream

+
+----
+EOM

  • expected.sub!(//, filename)
  • expected.sub!(//, $test_net_http_data)
  • start {|http|
  •  data.each{|k,v|v.rewind rescue nil} 
  •  req = Net::HTTP::Post.new('/') 
  •  req.set_form(data, 'multipart/form-data') 
  •  res = http.request req 
  •  body = res.body 
  •  header, _ = body.split(/\r\n\r\n/, 2) 
  •  assert_match(/\A--(?<boundary>\S+)/, body) 
  •  /\A--(?<boundary>\S+)/ =~ body 
  •  expected = expected.gsub(/<boundary>/, boundary) 
  •  assert_match(/^--(?<boundary>\S+)\r\n/, header) 
  •  assert_match( 
  •  /^Content-Disposition: form-data; name="file"; filename="#{filename}"\r\n/, 
  •  header) 
  •  assert_equal(expected, body) 
  •  data.each{|k,v|v.rewind rescue nil} 
  •  req['Transfer-Encoding'] = 'chunked' 
  •  res = http.request req 
  •  #assert_equal(expected, res.body) 
  • }
  • end
    end

class TestNetHTTP_version_1_1 < Test::Unit::TestCase
=end

Updated by naruse (Yui NARUSE) almost 15 years ago Actions #1

  • Status changed from Open to Closed
  • % Done changed from 0 to 100

=begin
This issue was solved with changeset r30188.
Yui, thank you for reporting this issue.
Your contribution to Ruby is greatly appreciated.
May Ruby be with you.

=end

Updated by nahi (Hiroshi Nakamura) over 14 years ago Actions #2 [ruby-dev:43846]

  • Status changed from Closed to Open

Naruse-san, would you please add an explanation of this feature to NEWS file?

Updated by nahi (Hiroshi Nakamura) over 14 years ago Actions #3 [ruby-dev:43847]

すいませんruby-devだった。orz

成瀬さん、NEWSになんか書きませんか。

Updated by naruse (Yui NARUSE) over 14 years ago Actions #4 [ruby-dev:43954]

  • Status changed from Open to Closed

r32241 で書きました。

Actions

Also available in: PDF Atom