Add a "--quiet" option to each image-acquiring command that currently
accepts a "--timeout" option, to allow the displaying of the download
URI and the progress dots to be inhibited.
This is particularly useful with "data:" URIs to inhibit the echoing
of the full data URI contents:
iPXE> imgfetch -n hw data:,hello%20world
data:,hello%20world... ok
iPXE>
vs.
iPXE> imgfetch -q -n hw data:,hello%20world
iPXE>
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Add a trivial ring buffer console that can be used to extract the most
recent 8kB of (non-UI) console output as the ${dmesg} setting.
This allows previous console output to be displayed after the screen
has been cleared, such as when a background picture has been loaded.
For example:
#!ipxe
console -p http://boot.ipxe.org/ipxe.png
show -q dmesg
It also allows console output to be captured and sent as part of an
HTTP POST, to allow for remote diagnostics. For example:
#!ipxe
params
param dmesg ${dmesg:base64}
imgfetch http://192.168.0.1/api/diags##params
The recorded console output may be cleared if necessary by clearing
the setting:
clear builtin/dmesg
The name ${dmesg} is chosen as being unlikely to collide with any
existing variables used in end-user scripts. A separate "dmesg"
command is not provided, but could easily be added if useful.
Note that iPXE supports recursive variable expansion in shell
commands. Typing an interactive command such as "echo ${dmesg}" or
"param dmesg ${dmesg}" is therefore a great way to exercise the memory
allocator to the point of exhaustion. Use "show -q dmesg" to show the
ring buffer contents.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Within application/x-www-form-urlencoded values, a "+" character needs
to be escaped to avoid its being interpreted as a space.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Allow the "show -q" command to be used to display a setting's value
without also showing its origin and type metadata.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Add support for "data:" URIs as defined in RFC 2397. These can be
used to construct image content under control of an iPXE script. For
example:
# Inject the message "Hello from iPXE" as /etc/motd
initrd -n motd data:,Hello%20from%20iPXE%0A /etc/motd
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Within the iPXE data transfer interface model, openers are fully
asynchronous and may not deliver any data until after the opener has
returned.
Provide a trivial openable data blob object (as a generalisation of
the "hello world" data transfer interface example code) that will
simply deliver a single fixed blob of data to its parent interface and
then close itself.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
The design of IMDSv2 within Alibaba Cloud is identical to AWS IMDSv2,
with the header names changed from "X-aws-ec2-*" to "X-aliyun-ecs-*".
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Use an HTTP PUT request to fetch a session token, and pass this token
value as a header when fetching the user-data script.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
A commonly requested feature is to allow a setting to be populated
with the contents of an HTTP response. This currently requires a
somewhat ugly workaround of having the HTTP endpoint generate an iPXE
executable script fragment that includes the "#!ipxe" shebang and the
relevant "set" command.
For HTTP endpoints that are under the end user's control, this
workaround is viable (though still ugly). For HTTP endpoints that are
outside the user's control (such as the AWS metadata endpoints), this
workaround cannot be used.
Add an "imgset" command that can be used to store downloaded content
directly into a setting.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
The design of IMDSv2 within both AWS and Alibaba Cloud requires the
client to obtain a temporary token via an HTTP PUT request. There is
no authentication on this request and there is no associated request
body: the requirement to use PUT exists solely to reduce the attack
surface for SSRF attacks (since vulnerable servers are much more
likely to be able to be tricked into issuing a GET request than a PUT
request).
iPXE can currently issue requests using HTTP GET (if the request body
is empty) or HTTP POST (if the request body includes form parameters).
There is no support for issuing a PUT request, or for allowing a
script to explicitly specify the HTTP method.
Add a "--method" option to the "params" command to allow an arbitrary
request method name to be specified, and use this as the HTTP request
method.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
If a named parameter block is created and then a URI is parsed that
attempts to use a nonexistent unnamed parameter block (or vice versa),
then the code in find_parameters() will currently call strcmp() with a
NULL argument, resulting in a read-only access to undefined memory.
Fix by calling strcmp() only for non-NULL names.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Some public clouds (such as AWS and Alibaba Cloud) allow for only a
single user metadata blob. The official iPXE cloud images will
attempt to download and boot from this user metadata, expecting it to
contain an iPXE script.
This works, but causes conflicts when another consumer (such as
cloud-init) also wants to use the same metadata blob. There are
workarounds (such as publishing the cloud-init script at an
alternative URI outside of the instance metadata service, and using
the iPXE script to direct cloud-init to use the alternative URI via
kernel command-line arguments), but these are cumbersome and may
weaken security since the alternative URI cannot provide the same
level of guaranteed access restrictions.
There is support within cloud-init for parsing a multipart MIME
archive, which may contain additional shell scripts, JSON data, etc,
alongside the cloud-init configuration itself. This is the standard
and documented method that cloud-init has chosen to solve the issue of
obtaining multiple data sources from a single user metadata blob.
Add support for multipart MIME as an archive image format from which
iPXE will extract the first body part that has the "text/x-ipxe" MIME
type. This allows the iPXE boot script to be placed alongside
cloud-init configuration within a single user metadata blob.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
The zlib and gzip test definitions are almost identical. Create a
single definition of an archive test to reduce duplication.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Ensure that the reset register write does not get reordered behind the
first PCI configuration space read that checks to see if the reset has
completed.
Debugged-by: Jaroslav Svoboda <multi.flexi@seznam.cz>
Tested-by: Jaroslav Svoboda <multi.flexi@seznam.cz>
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Using standard string functions for parsing text-based image formats
is currently cumbersome since there is no guaranteed NUL terminator,
and so code must laboriously keep track of the remaining image length
and use only those string functions that accept a length limit.
Ensure that the byte immediately following the image data is always a
NUL, thereby allowing all string functions to be used when parsing
images. Provide a "const char *text" pointer aliased to the image
data, to make it explicit that image data may always be treated as a
NUL-terminated string.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Define and use a data transfer buffer that is directly backed by an
image, rather than downloading into a umalloc()-based data transfer
buffer and then transferring ownership to the image.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Change the "bound" field from being a boolean flag to being a
reference to the server identity (i.e. the certificate) to which the
shared secret has been bound.
This reduces the chances for future bugs that could be caused by
potentially losing track of which identity has been bound, and also
provides a natural way to extend the field to be able to represent an
identity that has not yet been validated (as will be required for TLS
version 1.3 key exchange).
Add a check that the bound identity has been validated at the point of
sending our client Finished handshake. We must defer sending the
client Finished until validation has completed, to prevent the server
from sending application traffic until we are ready to receive it, and
so this provides a natural point at which we know that the bound
identity must have been validated.
Since the validity check is now deferred until the point of sending
the client Finished, and since commit 6ba010e ("[tls] Reject incorrect
server names before completing validation") already ensures that the
certificate must have the correct name, there is no need to extract
and store the certificate's public key separately after validation has
completed.
We also update the session key (and related parameters) only if the
bound identity has been validated. This creates an invariant that if
a server certificate is stored in the session then it is always
guaranteed to be valid, which simplifies reasoning about session
resumption flows.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
We currently verify the certificate name only after completing
validation of the certificate chain. Perform this check instead at
the point of parsing the Certificate record, to create an invariant
that the recorded server certificate always has the correct name (even
if not yet validated).
Signed-off-by: Michael Brown <mcb30@ipxe.org>
There should be no circumstance that leads to a session being resumed
without having a valid session resumption secret that was stored by a
successfully established previous connection.
As an additional layer of defence in depth, clear the "keyed" and
"bound" flags in the key schedule if it is ever resumed from an empty
session resumption secret.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
There should be no circumstance that leads to a session being resumed
without having a valid session resumption secret that was stored by a
successfully established previous connection.
As an additional layer of defence in depth, poison the initial session
resumption master secret so that a predictable all-zero secret can
never be used accidentally in future.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
A freshly initialised key derivation function master secret has the
"keyed" flag clear and so cannot accidentally be used to establish a
full TLS connection.
As an additional layer of defence in depth, poison the initial key
derivation master secret so that a predictable all-zero secret can
never be used accidentally in future.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
TLS already allows for several different paths to the establishment of
a shared secret channel. The shared secret may be generated by the
client and encrypted using RSA key transport, or negotiated as a
Diffie-Hellman shared secret (via FFDHE or ECDHE), or mutually agreed
to be restored from a previously saved session resumption secret.
TLS version 1.3 defines several new paths to exist alongside these:
ephemeral key exchange is moved to the ClientHello and ServerHello
messages, server identity is verified using a CertificateVerify
message (instead of a signed ServerKeyExchange or an encrypted
ClientKeyExchange), and session resumption is handled via a new
pre-shared key mechanism.
While TLS version 1.3 in isolation is substantially simpler and
cleaner than earlier versions, the requirement to support both new and
old versions in the same code comes with a significant complexity
cost.
Guard against the possibility of future bugs by defining two
properties for the key schedule:
- a "keyed" flag indicating that the key schedule actually holds some
shared secret key material (e.g. from ECDHE)
- a "bound" flag indicating that the shared secret key material in the
key schedule has been bound to the identity represented by the
server's certificate
These flags are updated when relevant key schedule events happen, and
validated before processing the server's Finished message. If we
somehow end up receiving a Finished message without having established
and authenticated a shared secret, this check prevents us from marking
the connection as ready for application data.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
A malicious server that immediately sends a Finished record (without
ever having sent a ServerHello) will currently cause tls_prf() to get
stuck in an infinite loop attempting to generate pseudorandom data
using the null digest algorithm.
Fix by checking that the key schedule digest size is non-zero
(i.e. that the digest is not the null digest) before attempting to
process the Finished record.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
When the TLS connection is closed by the underlying socket, the
closure alert will not be able to be sent. This currently results in
a harmless but mildly irritating error message when debugging is
enabled.
Fix by sending the closure alert only when we are actively choosing to
close the connection.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Ephemeral key exchange is currently handled as part of sending the
ClientKeyExchange handshake record, with almost entirely separate
implementations for DHE and ECDHE.
Create wrappers around the underlying key exchange algorithm to handle
the TLS-specific aspects (such as padding and stripping leading zeros
for DHE), and use these for both DHE and ECDHE key exchange.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Requests for metadata within Google Compute Engine require a custom
HTTP header "Metadata-Flavor: Google" to guard against Server-Side
Request Forgery (SSRF) attacks.
Support for this was originally implemented in 2017 using a custom
HTTP request header generator that added this header to any requests
made to metadata.google.com.
In commit 96bb6ba ("[params] Allow for arbitrary HTTP request headers
to be specified"), iPXE gained the ability to generate arbitrary HTTP
headers via the "param" command.
Remove the custom HTTP request header generator, enable the "param"
command for cloud builds, and update the embedded script for Google
Compute Engine to construct the required Metadata-Flavor header.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Calls to the key derivation function tls_prf() currently have to pass
the relevant secret (i.e. the pre-master secret or the master secret)
as a parameter.
Restructure to more closely match the design of the TLS version 1.3
key schedule, which maintains a single running secret (which we choose
to name the "key derivation function master secret") that is always
implicitly used as the secret for key expansion.
The secret value is currently used by tls_prf() only as the key to
hmac_init(). We can therefore use hmac_key() to reduce the secret to
a fixed-length value. (The TLS master secret is already a fixed 48
bytes, but the pre-master secret may be any length.)
The fixed length of the secret is dependent upon the protocol version
and the cipher suite digest algorithm. For TLS version 1.2, the
length is the HMAC key size for the digest algorithm. For TLS version
1.1, which uses separate invocations of HMAC-MD5 and HMAC-SHA1, the
length is the sum of the HMAC-MD5 and HMAC-SHA1 key sizes. (For TLS
version 1.3, the length will be the HKDF key size, i.e. the output
size of the digest algorithm.)
To avoid introducing some very messy memory allocation code paths, we
continue to use a fixed size of 48 bytes for the resumption master
secret stored in the TLS session. This is sufficient to hold the
48-byte raw master secret for TLS version 1.2 and earlier, and will
also be sufficient to hold the HKDF Derive-Secret output for the
longest supported digest algorithm (SHA-384) in TLS version 1.3.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
When resuming a session in TLS versions 1.2 and earlier, the master
secret is simply reused. The value stored in the session is therefore
the same as the master secret used in the connection.
For TLS version 1.3, there is a separate concept of a "resumption
master secret" that is derived from the original connection's master
secret, and from which the resumed connection's new master secret will
be derived.
Rename the session master_secret to resumption_master_secret to
clarify this separation.
Resume use of the master secret (i.e. copy the secret from the session
to the connection) only after the cipher suite has been selected, to
reduce differences with the expected flow for TLS version 1.3.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
When using QEMU with the KVM accelerator, MMIO writes to the queue
doorbell register are likely to hit an ioeventfd region. KVM will
signal readiness on the ioeventfd file descriptor (which will
eventually wake up the QEMU userspace process to handle the MMIO
write) and then immediately resume execution of the iPXE guest.
This can result in high latencies in processing submitted descriptors.
With the small transmit queue fill level used by iPXE, this can easily
overrun the transmit queue and result in large numbers of dropped
transmissions.
Increase the transmit queue fill level to utilise the whole queue if
needed, and use the transmission deferral mechanism to avoid dropping
packets when high latencies occur during operation.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
The queue size calculations currently do not take into account the
fact that each packet requires a pair of descriptors. If the device
happens to present an extremely small queue (16 descriptors for Q0, 32
descriptors for Q1) then this will result in the driver submitting
descriptors beyond the queue's descriptor count. This does not result
in any invalid memory accesses (since the descriptor ring length is
rounded up to a 4kB boundary), but does result in an unusable network
device.
Fix by scaling the packet count and descriptor count values as
required.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
The digest algorithm selected by the cipher suite is used for both
calculating the handshake digest and as the tls_prf() key derivation
function digest algorithm. (In TLS version 1.3, it will similarly be
used as the HKDF digest algorithm.)
Move the digest algorithm selection within the key schedule, to
clarify that the scope of this digest algorithm is wider than solely
being used to calculate the handshake digest.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
The term "key" has a large number of uses in the context of TLS.
Reduce opportunities for confusion by renaming tls_key_init() and
tls_key_reset() to less generic names.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
The hybrid PRF used in TLS version 1.1 and earlier does not use HMAC
with the hybrid MD5+SHA1 algorithm: it uses separate invocations of
HMAC-MD5 and HMAC-SHA1.
Using hmac_keysize(&md5_sha1_algorithm) would produce a size too small
to hold the combined HMAC-MD5 and HMAC-SHA1 keys. One option would be
to set the (currently unused) MD5+SHA1 block size to 128, thereby
ensuring that hmac_keysize() would happen to return a length large
enough to hold both HMAC keys. This would avoid the need to
special-case the MD5+SHA1 algorithm when calculating the required HMAC
key size for the PRF, but would inevitably cause confusion in future.
Set the MD5+SHA1 block size to 64 (since both algorithms have the same
underlying block size, and this would therefore produce the "correct"
result if anything were ever to use HMAC directly with the hybrid
MD5+SHA1 algorithm), and define a separate structure for holding the
separated HMAC keys used by the PRF in TLS version 1.1 and earlier.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
The minimum supported TLS version is already configurable via
TLS_VERSION_MIN in config/crypto.h, but changing the maximum TLS
version currently requires editing the source code proper. This makes
it cumbersome to test older TLS versions, and therefore increases the
chances that support for older versions will end up breaking as new
features are added.
Move TLS_VERSION_MAX from include/ipxe/tls.h to config/crypto.h to
ease the process of testing older TLS versions.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Commit efa9515 ("[tls] Split out hybrid MD5+SHA1 algorithm used in TLS
version 1.1") accidentally removed the empty RSA digestInfo prefix
required for verifying DHE and ECDHE ServerKeyExchange messages when
using TLS version 1.1. (Non-ephemeral cipher suites using RSA key
transport would still work, since the digestInfo is required only for
signatures, not for encryption/decryption.)
Fix by restoring the dummy digestInfo prefix for MD5+SHA1.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
As described in commit 05cb930 ("[build] Extend default configuration
for non-BIOS builds"), the default configuration for EFI needs to
allow for the unfortunate fact that users will not be able to rebuild
the Secure Boot binaries for themselves.
The keyboard map currently defaults to "us" (i.e. no keyboard
remapping) on all platforms. Switch to using the "dynamic" keyboard
map by default for EFI platforms.
Do not use the "dynamic" keyboard map by default on Linux platforms
(where the input character read by iPXE has already passed through the
host's keyboard mapping) or on RISC-V SBI (where input is expected to
come via a serial port rather than a directly attached keyboard).
Requested-by: Simon Fonteneau <blog@lesfourmisduweb.org>
Signed-off-by: Michael Brown <mcb30@ipxe.org>
An HMAC key can always be reduced to the block size of the underlying
digest algorithm. Provide hmac_key() that can be used to perform this
reduction, and hmac_init_key() as a way to initialise an HMAC digest
operation from a previously reduced key.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
The S/390 architecture provides instructions to read the Time-of-Day
(TOD) clock, which increments at a well-defined rate regardless of the
underlying physical clock speed, and has an epoch that starts from
zero at the beginning of the 20th century.
Use this clock to provide both interval timing (i.e. udelay() and
currticks()) and the wall-clock time source. For short interval
timing, we choose to save on code size by treating one millisecond as
1024 microseconds.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
The "prno" instruction available on newer CPUs provides a hardware
True Random Number Generator (TRNG) that can be used as an entropy
source, similar to the x86 "rdrand" instruction.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Remove the now-unused implementation of DHE that requires explicit
group parameters, since we now use a standalone key exchange algorithm
abstraction instead.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Now that key exchange algorithms are allowed to fail to construct a
shared public key, we can allocate the temporary working space for
FFDHE calculations on demand rather than using a static buffer.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Commit 70d63be ("[crypto] Add RFC 3526 FFDHE key exchange algorithms")
defined FFDHE_LEN as a fixed value (rather than deriving it from the
stored length of the Euler constant) and accidentally expressed it as
a bit length rather than a byte length, resulting in substantial
amounts of wasted space.
Fix the maximum length of the modulus and add static assertions to
ensure that the two constants are exactly the required size for this
length.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
All of our current digest algorithms (MD4, MD5, SHA-1, and the SHA-2
family) use a Merkle-Damgård construction, with only the compression
function, the initial digest values, the field sizes, and the
endianness differing between algorithms.
Provide a common implementation for Merkle-Damgård hash algorithms to
reduce code size. Values are now held as host-endian quantities, with
any swapping performed byte-by-byte as data is accumulated (using a
compile-time constant that is XORed with the byte index).
For the SHA family of algorithms, the values w[] are now calculated
iteratively as we progress through the main loop: this substantially
reduces the stack space required for the compression function.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Michael Brown
