The Content-Transfer-Encoding header is optional: if not present then
the default "7bit" encoding should be assumed. iPXE already includes
logic to set a default encoding name, but the default encoding name
then fails to match against any entries in the known encodings list
since it is terminated with a NUL (rather than with the semicolon or
whitespace character that would terminate the encoding name found
within a Content-Transfer-Encoding header).
Fix by removing the default encoding name and instead treating a NULL
encoding name as indicating that the default encoding should be used,
and add a test case that omits the Content-Transfer-Encoding header.
Reported-by: Huzaifa Ali Zar <zar@amazon.com>
Signed-off-by: Michael Brown <mcb30@ipxe.org>
When specifying the content of a test image (e.g. a MIME archive file)
using C string literals, there is no easy way to indicate that the
terminating NUL should be excluded from the byte array.
Provide BINFILE(), TEXTFILE(), and FILE_ARRAY() helper macros that can
be used to simplify the initialisation of a byte array passed as
either a raw byte value list or a string literal. For example:
#define TEST_CASE( name, file ) do { \
static uint8_t name ## _bytes FILE_ARRAY ( file ); \
... \
} while ( 0 )
TEST_CASE ( test1, BINFILE ( 0x68, 0x65, 0x6c, 0x6c, 0x6f ) );
TEST_CASE ( test2, TEXTFILE ( "hello" ) );
Both of the above TEST_CASE() lines will end up producing a five-byte
array:
static uint8_t test1_bytes[] = { 0x68, 0x65, 0x6c, 0x6c, 0x6f };
static uint8_t test2_bytes[5] = "hello";
This allows us to remove the stray NUL that otherwise appears at the
end of any test images that are specified using string literals.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Commit 3662065 ("[dns] Use all configured DNS servers") changed the
logic from opening a single defined nameserver address to opening an
unspecified peer socket address and then specifying the full peer
address for each transmitted packet.
The peer socket address was left unspecified by passing a null pointer
to xfer_open_socket(). This is supported by the UDP socket opener,
but technically violates the internal API (which allows the local
socket address to be a null pointer, but not the peer socket address).
In particular, in a debug build using DEBUG=open, the debug code will
itself dereference the peer address pointer.
Fix by embedding the name server socket address within the DNS request
structure, and passing this to xfer_open_socket().
Signed-off-by: Michael Brown <mcb30@ipxe.org>
With no current working URI, even a fully resolved URI may not have a
scheme. Attempting to open such a URI will currently result in
xfer_uri_opener() calling strcasecmp() with a null pointer. On a
system that guards against null pointer dereferences, this will result
in a segfault (or the equivalent, such as a Synchronous Exception on
arm64 UEFI).
Fix by checking that the URI is absolute (i.e. has a scheme) before
calling xfer_uri_opener(), as is already done elsewhere.
Reported-by: Matt Fleming <matt@readmodwrite.com>
Signed-off-by: Michael Brown <mcb30@ipxe.org>
As of commit 433a8f5 ("[tls] Retain a reference in the key schedule to
the bound identity"), the act of binding the server identity is
logically separated from the act of validating the server identity.
We may therefore bind the server identity (by verifying the signature
over the Diffie-Hellman parameters) and agree the ephemeral shared
secret immediately upon receiving the ServerKeyExchange record, rather
than deferring the verification until we have a validated identity.
This provides a closer match to the flow required for TLS version 1.3,
where the ephemeral shared secret is used for all messages after
ServerHello, and so must always be agreed prior to validation.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Add a "--quiet" option to each image-acquiring command that currently
accepts a "--timeout" option, to allow the displaying of the download
URI and the progress dots to be inhibited.
This is particularly useful with "data:" URIs to inhibit the echoing
of the full data URI contents:
iPXE> imgfetch -n hw data:,hello%20world
data:,hello%20world... ok
iPXE>
vs.
iPXE> imgfetch -q -n hw data:,hello%20world
iPXE>
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Add a trivial ring buffer console that can be used to extract the most
recent 8kB of (non-UI) console output as the ${dmesg} setting.
This allows previous console output to be displayed after the screen
has been cleared, such as when a background picture has been loaded.
For example:
#!ipxe
console -p http://boot.ipxe.org/ipxe.png
show -q dmesg
It also allows console output to be captured and sent as part of an
HTTP POST, to allow for remote diagnostics. For example:
#!ipxe
params
param dmesg ${dmesg:base64}
imgfetch http://192.168.0.1/api/diags##params
The recorded console output may be cleared if necessary by clearing
the setting:
clear builtin/dmesg
The name ${dmesg} is chosen as being unlikely to collide with any
existing variables used in end-user scripts. A separate "dmesg"
command is not provided, but could easily be added if useful.
Note that iPXE supports recursive variable expansion in shell
commands. Typing an interactive command such as "echo ${dmesg}" or
"param dmesg ${dmesg}" is therefore a great way to exercise the memory
allocator to the point of exhaustion. Use "show -q dmesg" to show the
ring buffer contents.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Within application/x-www-form-urlencoded values, a "+" character needs
to be escaped to avoid its being interpreted as a space.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Allow the "show -q" command to be used to display a setting's value
without also showing its origin and type metadata.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Add support for "data:" URIs as defined in RFC 2397. These can be
used to construct image content under control of an iPXE script. For
example:
# Inject the message "Hello from iPXE" as /etc/motd
initrd -n motd data:,Hello%20from%20iPXE%0A /etc/motd
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Within the iPXE data transfer interface model, openers are fully
asynchronous and may not deliver any data until after the opener has
returned.
Provide a trivial openable data blob object (as a generalisation of
the "hello world" data transfer interface example code) that will
simply deliver a single fixed blob of data to its parent interface and
then close itself.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
The design of IMDSv2 within Alibaba Cloud is identical to AWS IMDSv2,
with the header names changed from "X-aws-ec2-*" to "X-aliyun-ecs-*".
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Use an HTTP PUT request to fetch a session token, and pass this token
value as a header when fetching the user-data script.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
A commonly requested feature is to allow a setting to be populated
with the contents of an HTTP response. This currently requires a
somewhat ugly workaround of having the HTTP endpoint generate an iPXE
executable script fragment that includes the "#!ipxe" shebang and the
relevant "set" command.
For HTTP endpoints that are under the end user's control, this
workaround is viable (though still ugly). For HTTP endpoints that are
outside the user's control (such as the AWS metadata endpoints), this
workaround cannot be used.
Add an "imgset" command that can be used to store downloaded content
directly into a setting.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
The design of IMDSv2 within both AWS and Alibaba Cloud requires the
client to obtain a temporary token via an HTTP PUT request. There is
no authentication on this request and there is no associated request
body: the requirement to use PUT exists solely to reduce the attack
surface for SSRF attacks (since vulnerable servers are much more
likely to be able to be tricked into issuing a GET request than a PUT
request).
iPXE can currently issue requests using HTTP GET (if the request body
is empty) or HTTP POST (if the request body includes form parameters).
There is no support for issuing a PUT request, or for allowing a
script to explicitly specify the HTTP method.
Add a "--method" option to the "params" command to allow an arbitrary
request method name to be specified, and use this as the HTTP request
method.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
If a named parameter block is created and then a URI is parsed that
attempts to use a nonexistent unnamed parameter block (or vice versa),
then the code in find_parameters() will currently call strcmp() with a
NULL argument, resulting in a read-only access to undefined memory.
Fix by calling strcmp() only for non-NULL names.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Some public clouds (such as AWS and Alibaba Cloud) allow for only a
single user metadata blob. The official iPXE cloud images will
attempt to download and boot from this user metadata, expecting it to
contain an iPXE script.
This works, but causes conflicts when another consumer (such as
cloud-init) also wants to use the same metadata blob. There are
workarounds (such as publishing the cloud-init script at an
alternative URI outside of the instance metadata service, and using
the iPXE script to direct cloud-init to use the alternative URI via
kernel command-line arguments), but these are cumbersome and may
weaken security since the alternative URI cannot provide the same
level of guaranteed access restrictions.
There is support within cloud-init for parsing a multipart MIME
archive, which may contain additional shell scripts, JSON data, etc,
alongside the cloud-init configuration itself. This is the standard
and documented method that cloud-init has chosen to solve the issue of
obtaining multiple data sources from a single user metadata blob.
Add support for multipart MIME as an archive image format from which
iPXE will extract the first body part that has the "text/x-ipxe" MIME
type. This allows the iPXE boot script to be placed alongside
cloud-init configuration within a single user metadata blob.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
The zlib and gzip test definitions are almost identical. Create a
single definition of an archive test to reduce duplication.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Ensure that the reset register write does not get reordered behind the
first PCI configuration space read that checks to see if the reset has
completed.
Debugged-by: Jaroslav Svoboda <multi.flexi@seznam.cz>
Tested-by: Jaroslav Svoboda <multi.flexi@seznam.cz>
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Using standard string functions for parsing text-based image formats
is currently cumbersome since there is no guaranteed NUL terminator,
and so code must laboriously keep track of the remaining image length
and use only those string functions that accept a length limit.
Ensure that the byte immediately following the image data is always a
NUL, thereby allowing all string functions to be used when parsing
images. Provide a "const char *text" pointer aliased to the image
data, to make it explicit that image data may always be treated as a
NUL-terminated string.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Define and use a data transfer buffer that is directly backed by an
image, rather than downloading into a umalloc()-based data transfer
buffer and then transferring ownership to the image.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Change the "bound" field from being a boolean flag to being a
reference to the server identity (i.e. the certificate) to which the
shared secret has been bound.
This reduces the chances for future bugs that could be caused by
potentially losing track of which identity has been bound, and also
provides a natural way to extend the field to be able to represent an
identity that has not yet been validated (as will be required for TLS
version 1.3 key exchange).
Add a check that the bound identity has been validated at the point of
sending our client Finished handshake. We must defer sending the
client Finished until validation has completed, to prevent the server
from sending application traffic until we are ready to receive it, and
so this provides a natural point at which we know that the bound
identity must have been validated.
Since the validity check is now deferred until the point of sending
the client Finished, and since commit 6ba010e ("[tls] Reject incorrect
server names before completing validation") already ensures that the
certificate must have the correct name, there is no need to extract
and store the certificate's public key separately after validation has
completed.
We also update the session key (and related parameters) only if the
bound identity has been validated. This creates an invariant that if
a server certificate is stored in the session then it is always
guaranteed to be valid, which simplifies reasoning about session
resumption flows.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
We currently verify the certificate name only after completing
validation of the certificate chain. Perform this check instead at
the point of parsing the Certificate record, to create an invariant
that the recorded server certificate always has the correct name (even
if not yet validated).
Signed-off-by: Michael Brown <mcb30@ipxe.org>
There should be no circumstance that leads to a session being resumed
without having a valid session resumption secret that was stored by a
successfully established previous connection.
As an additional layer of defence in depth, clear the "keyed" and
"bound" flags in the key schedule if it is ever resumed from an empty
session resumption secret.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
There should be no circumstance that leads to a session being resumed
without having a valid session resumption secret that was stored by a
successfully established previous connection.
As an additional layer of defence in depth, poison the initial session
resumption master secret so that a predictable all-zero secret can
never be used accidentally in future.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
A freshly initialised key derivation function master secret has the
"keyed" flag clear and so cannot accidentally be used to establish a
full TLS connection.
As an additional layer of defence in depth, poison the initial key
derivation master secret so that a predictable all-zero secret can
never be used accidentally in future.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
TLS already allows for several different paths to the establishment of
a shared secret channel. The shared secret may be generated by the
client and encrypted using RSA key transport, or negotiated as a
Diffie-Hellman shared secret (via FFDHE or ECDHE), or mutually agreed
to be restored from a previously saved session resumption secret.
TLS version 1.3 defines several new paths to exist alongside these:
ephemeral key exchange is moved to the ClientHello and ServerHello
messages, server identity is verified using a CertificateVerify
message (instead of a signed ServerKeyExchange or an encrypted
ClientKeyExchange), and session resumption is handled via a new
pre-shared key mechanism.
While TLS version 1.3 in isolation is substantially simpler and
cleaner than earlier versions, the requirement to support both new and
old versions in the same code comes with a significant complexity
cost.
Guard against the possibility of future bugs by defining two
properties for the key schedule:
- a "keyed" flag indicating that the key schedule actually holds some
shared secret key material (e.g. from ECDHE)
- a "bound" flag indicating that the shared secret key material in the
key schedule has been bound to the identity represented by the
server's certificate
These flags are updated when relevant key schedule events happen, and
validated before processing the server's Finished message. If we
somehow end up receiving a Finished message without having established
and authenticated a shared secret, this check prevents us from marking
the connection as ready for application data.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
A malicious server that immediately sends a Finished record (without
ever having sent a ServerHello) will currently cause tls_prf() to get
stuck in an infinite loop attempting to generate pseudorandom data
using the null digest algorithm.
Fix by checking that the key schedule digest size is non-zero
(i.e. that the digest is not the null digest) before attempting to
process the Finished record.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
When the TLS connection is closed by the underlying socket, the
closure alert will not be able to be sent. This currently results in
a harmless but mildly irritating error message when debugging is
enabled.
Fix by sending the closure alert only when we are actively choosing to
close the connection.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Ephemeral key exchange is currently handled as part of sending the
ClientKeyExchange handshake record, with almost entirely separate
implementations for DHE and ECDHE.
Create wrappers around the underlying key exchange algorithm to handle
the TLS-specific aspects (such as padding and stripping leading zeros
for DHE), and use these for both DHE and ECDHE key exchange.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Requests for metadata within Google Compute Engine require a custom
HTTP header "Metadata-Flavor: Google" to guard against Server-Side
Request Forgery (SSRF) attacks.
Support for this was originally implemented in 2017 using a custom
HTTP request header generator that added this header to any requests
made to metadata.google.com.
In commit 96bb6ba ("[params] Allow for arbitrary HTTP request headers
to be specified"), iPXE gained the ability to generate arbitrary HTTP
headers via the "param" command.
Remove the custom HTTP request header generator, enable the "param"
command for cloud builds, and update the embedded script for Google
Compute Engine to construct the required Metadata-Flavor header.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Calls to the key derivation function tls_prf() currently have to pass
the relevant secret (i.e. the pre-master secret or the master secret)
as a parameter.
Restructure to more closely match the design of the TLS version 1.3
key schedule, which maintains a single running secret (which we choose
to name the "key derivation function master secret") that is always
implicitly used as the secret for key expansion.
The secret value is currently used by tls_prf() only as the key to
hmac_init(). We can therefore use hmac_key() to reduce the secret to
a fixed-length value. (The TLS master secret is already a fixed 48
bytes, but the pre-master secret may be any length.)
The fixed length of the secret is dependent upon the protocol version
and the cipher suite digest algorithm. For TLS version 1.2, the
length is the HMAC key size for the digest algorithm. For TLS version
1.1, which uses separate invocations of HMAC-MD5 and HMAC-SHA1, the
length is the sum of the HMAC-MD5 and HMAC-SHA1 key sizes. (For TLS
version 1.3, the length will be the HKDF key size, i.e. the output
size of the digest algorithm.)
To avoid introducing some very messy memory allocation code paths, we
continue to use a fixed size of 48 bytes for the resumption master
secret stored in the TLS session. This is sufficient to hold the
48-byte raw master secret for TLS version 1.2 and earlier, and will
also be sufficient to hold the HKDF Derive-Secret output for the
longest supported digest algorithm (SHA-384) in TLS version 1.3.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
When resuming a session in TLS versions 1.2 and earlier, the master
secret is simply reused. The value stored in the session is therefore
the same as the master secret used in the connection.
For TLS version 1.3, there is a separate concept of a "resumption
master secret" that is derived from the original connection's master
secret, and from which the resumed connection's new master secret will
be derived.
Rename the session master_secret to resumption_master_secret to
clarify this separation.
Resume use of the master secret (i.e. copy the secret from the session
to the connection) only after the cipher suite has been selected, to
reduce differences with the expected flow for TLS version 1.3.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
When using QEMU with the KVM accelerator, MMIO writes to the queue
doorbell register are likely to hit an ioeventfd region. KVM will
signal readiness on the ioeventfd file descriptor (which will
eventually wake up the QEMU userspace process to handle the MMIO
write) and then immediately resume execution of the iPXE guest.
This can result in high latencies in processing submitted descriptors.
With the small transmit queue fill level used by iPXE, this can easily
overrun the transmit queue and result in large numbers of dropped
transmissions.
Increase the transmit queue fill level to utilise the whole queue if
needed, and use the transmission deferral mechanism to avoid dropping
packets when high latencies occur during operation.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
The queue size calculations currently do not take into account the
fact that each packet requires a pair of descriptors. If the device
happens to present an extremely small queue (16 descriptors for Q0, 32
descriptors for Q1) then this will result in the driver submitting
descriptors beyond the queue's descriptor count. This does not result
in any invalid memory accesses (since the descriptor ring length is
rounded up to a 4kB boundary), but does result in an unusable network
device.
Fix by scaling the packet count and descriptor count values as
required.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
The digest algorithm selected by the cipher suite is used for both
calculating the handshake digest and as the tls_prf() key derivation
function digest algorithm. (In TLS version 1.3, it will similarly be
used as the HKDF digest algorithm.)
Move the digest algorithm selection within the key schedule, to
clarify that the scope of this digest algorithm is wider than solely
being used to calculate the handshake digest.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
The term "key" has a large number of uses in the context of TLS.
Reduce opportunities for confusion by renaming tls_key_init() and
tls_key_reset() to less generic names.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
The hybrid PRF used in TLS version 1.1 and earlier does not use HMAC
with the hybrid MD5+SHA1 algorithm: it uses separate invocations of
HMAC-MD5 and HMAC-SHA1.
Using hmac_keysize(&md5_sha1_algorithm) would produce a size too small
to hold the combined HMAC-MD5 and HMAC-SHA1 keys. One option would be
to set the (currently unused) MD5+SHA1 block size to 128, thereby
ensuring that hmac_keysize() would happen to return a length large
enough to hold both HMAC keys. This would avoid the need to
special-case the MD5+SHA1 algorithm when calculating the required HMAC
key size for the PRF, but would inevitably cause confusion in future.
Set the MD5+SHA1 block size to 64 (since both algorithms have the same
underlying block size, and this would therefore produce the "correct"
result if anything were ever to use HMAC directly with the hybrid
MD5+SHA1 algorithm), and define a separate structure for holding the
separated HMAC keys used by the PRF in TLS version 1.1 and earlier.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
The minimum supported TLS version is already configurable via
TLS_VERSION_MIN in config/crypto.h, but changing the maximum TLS
version currently requires editing the source code proper. This makes
it cumbersome to test older TLS versions, and therefore increases the
chances that support for older versions will end up breaking as new
features are added.
Move TLS_VERSION_MAX from include/ipxe/tls.h to config/crypto.h to
ease the process of testing older TLS versions.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Commit efa9515 ("[tls] Split out hybrid MD5+SHA1 algorithm used in TLS
version 1.1") accidentally removed the empty RSA digestInfo prefix
required for verifying DHE and ECDHE ServerKeyExchange messages when
using TLS version 1.1. (Non-ephemeral cipher suites using RSA key
transport would still work, since the digestInfo is required only for
signatures, not for encryption/decryption.)
Fix by restoring the dummy digestInfo prefix for MD5+SHA1.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
As described in commit 05cb930 ("[build] Extend default configuration
for non-BIOS builds"), the default configuration for EFI needs to
allow for the unfortunate fact that users will not be able to rebuild
the Secure Boot binaries for themselves.
The keyboard map currently defaults to "us" (i.e. no keyboard
remapping) on all platforms. Switch to using the "dynamic" keyboard
map by default for EFI platforms.
Do not use the "dynamic" keyboard map by default on Linux platforms
(where the input character read by iPXE has already passed through the
host's keyboard mapping) or on RISC-V SBI (where input is expected to
come via a serial port rather than a directly attached keyboard).
Requested-by: Simon Fonteneau <blog@lesfourmisduweb.org>
Signed-off-by: Michael Brown <mcb30@ipxe.org>
An HMAC key can always be reduced to the block size of the underlying
digest algorithm. Provide hmac_key() that can be used to perform this
reduction, and hmac_init_key() as a way to initialise an HMAC digest
operation from a previously reduced key.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
The S/390 architecture provides instructions to read the Time-of-Day
(TOD) clock, which increments at a well-defined rate regardless of the
underlying physical clock speed, and has an epoch that starts from
zero at the beginning of the 20th century.
Use this clock to provide both interval timing (i.e. udelay() and
currticks()) and the wall-clock time source. For short interval
timing, we choose to save on code size by treating one millisecond as
1024 microseconds.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
The "prno" instruction available on newer CPUs provides a hardware
True Random Number Generator (TRNG) that can be used as an entropy
source, similar to the x86 "rdrand" instruction.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Guanghua Zhang
