Retrieving XML from Web Services

Yorai Aminov
Thursday, August 05, 2004

Download the code for this article

One of the areas in which the .NET framework excels is creating and using Web services. A Web service is basically an object, hosted by a web server, communicating with its client using SOAP. SOAP is based on XML, and runs over HTTP. Developers are spared the gory details by the .NET framework, which provides classes implementing both SOAP servers and clients.

While .NET provides nice, easy-to-use wrappers around Web services, there are times when you need to see exactly what goes on behind the scenes. If a remote service that is not under your control returns unexpected results, you may want to examine the actual XML returned by that service. In this article, I'll show how this can be done in Delphi for .NET.

One way of trapping the actual XML passed between servers and clients is to use a proxy. An extremely nice (and free) tool is ProxyTrace - a simple HTTP proxy that can trace all communication between SOAP clients and servers. There are cases, however, when using a proxy is not enough - for example, when accessing certain secure Web services or when the proxy causes a timeout. In these cases, you have to write code that accesses the actual XML.

Web References

To consume a Web service in Delphi (or in Visual Studio .NET), simply select "Add Web Reference" from the Project menu. The IDE will then present a dialog box where you can select a WSDL file or import the interface directly from a running service (you can also import a web reference using the WSDL.EXE utility in the .NET Framework SDK, but the tool doesn't support Delphi directly). The framework will generate a proxy class you can use to access the Web service.

If you look at the source code for such a proxy class, you'll see that all (synchronous) methods are just wrappers that call the Invoke method. Invoke handles all the details of creating a SOAP envelope, placing it in a web request, sending the request to the server, retrieving the results, and converting the results into types and values your code can use. Invoke is implemented in the SoapHttpClientProtocol class, which is the base class for all SOAP client proxies.

Invoke retrieves the actual XML from a WebResponse object, which encapsulates a response received from a web server. The WebResponse object is returned by the GetWebResponse method of the SoapHttpClientProtocol class. The actual XML is accessible using a stream, which can be retrieved by calling the WebResponse's GetResponseStream method.

Reading the Response Stream

It seems, therefore, that retrieving the responce content is fairly straightforward: override the proxy's GetWebResponse method and read the response stream returned by GetResponseStream. This seems to work fine. Unfortunately, if you grab the response stream, the Web service proxy stops working.

The problem is that SoapHttpClientProtocol expects the response stream's position to be at the stream's beginning, and once the stream is read its position is moved to the end of the stream. This wouldn't be a problem, except that HttpWebResponse (the WebResponse descendant created by SoapHttpClientProtocol) creates an internal stream of type System.Net.ConnectStream, and returns a reference to it whenever the GetResponseStream method is called. The problem is that ConnectStream does not support seeking. Once you read it, its position can never be set back.

Custom Web Responses

The solution is to create a custom WebResponse descendant that is capable of accessing the real response stream, while still returning a valid response stream in the GetResponseStream method:

function ExposedHttpWebResponse.GetResponseStream: Stream;
var
  ContentStream: Stream;
  Reader: StreamReader;
  ReadData: array of Byte;
  ReadBytes: Integer;
begin
  ContentStream := response.GetResponseStream;
 
  Result := MemoryStream.Create;
 
  SetLength(ReadData, 1024);
  repeat
    ReadBytes := ContentStream.Read(ReadData, 0, 1024);
    Result.Write(ReadData, 0, ReadBytes);
  until ReadBytes < 1024;
 
  ContentStream.Close;
  response.Close;
 
  Reader := StreamReader.Create(Result);
  Result.Seek(0, SeekOrigin.Begin);
  responseContent := Reader.ReadToEnd;
  Result.Seek(0, SeekOrigin.Begin);
end;

The response and responseContent variables used by the method are private fields of the ExposedHttpWebResponse class. The response field is initialized in the constructor by calling the GetResponse method of the WebRequest class (a WebRequest object is passed to the proxy's GetWebResponse method). The responseContent field is used to hold the contents of the response stream, and will be made accessible through a read-only property.

ExposedHttpWebResponse is a basically a wrapper class around a real HttpWebResponse object. To complete its implementation, some additional methods and properties must be added, but they are thin wrappers that expose the methods and properties of HttpWebResponse.

Wrapping

All that's left is to override the proxy's GetWebResponse method to return an ExposedHttpWebResponse object, and store the result so we can retrieve the response XML. The ExposedSoapClientProtocol class is a SoapHttpClientProtocol descendant that does just that:

type
  ExposedSoapClientProtocol = class(System.Web.Services.Protocols.SoapHttpClientProtocol)
  private
    lastResponse: WebResponse;
  strict protected
    function GetWebResponse(request: WebRequest): WebResponse; override;
  public
    function GetResponseXML: string;
  end;
 
implementation
 
function ExposedSoapClientProtocol.GetWebResponse(request: WebRequest): WebResponse;
begin
  lastResponse := ExposedHttpWebResponse.Create(request);
  Result := lastResponse;
end;
 
function ExposedSoapClientProtocol.GetResponseXML: string;
begin
  if Assigned(lastResponse) then
    Result := (lastResponse as ExposedHttpWebResponse).Content
  else
    Result := '';
end;

The ExposedService unit contains the ExposedSoapClientProtocol class (and the private ExposedHttpWebResponse class). To use it, simply add the unit to your web reference unit's uses clause, and have the proxy class descend from ExposedSoapClientProtocol instead of SoapHttpClientProtocol. You can then call the GetResponseXML at any time to retrieve the last XML response received by the client.

Conclusion

This article presented a method of programmatically retrieving the XML returned by a Web service to its client proxy. With very little effort, you can extend the code and save the content of requests sent to the Web service.

The .NET Framework does a great job at abstracting the tasks of creating and consuming Web services, but makes it difficult to get "closer to the metal". The techniques demonstrated here can be used to access the underlying data streams, thus gaining a better knowledge of the processes involved - and even obtaining a higher level of control over them.