[http] Work around stateful authentication schemes

As pointedly documented in RFC7230 section 2.3, HTTP is a stateless
protocol: each request message can be understood in isolation from any
other requests or responses.  Various authentication schemes such as
NTLM break this fundamental property of HTTP and rely on the same TCP
connection being reused.

Work around these broken authentication schemes by ensuring that the
most recently pooled connection is reused for the subsequent
authentication retry.

Reported-by: Andreas Hammarskjöld <junior@2PintSoftware.com>
Tested-by: Andreas Hammarskjöld <junior@2PintSoftware.com>
Signed-off-by: Michael Brown <mcb30@ipxe.org>

src/net/tcp/httpcore.c

@@ -778,6 +778,18 @@ static int http_transfer_complete ( struct http_transaction *http ) {
 	http->len = 0;
 	assert ( http->remaining == 0 );
 
+	/* Retry immediately if applicable.  We cannot rely on an
+	 * immediate timer expiry, since certain Microsoft-designed
+	 * HTTP extensions such as NTLM break the fundamentally
+	 * stateless nature of HTTP and rely on the same connection
+	 * being reused for authentication.  See RFC7230 section 2.3
+	 * for further details.
+	 */
+	if ( ! http->response.retry_after ) {
+		http_reopen ( http );
+		return 0;
+	}
+
 	/* Start timer to initiate retry */
 	DBGC2 ( http, "HTTP %p retrying after %d seconds\n",
 		http, http->response.retry_after );