Improving HoloLens internal PCIe data link transmission: Microsoft shares AR/VR patents

(Nweon December 27, 2023) A wired connection is often called a "hard link", which physically connects one node to another node. One wired connection is PCIe. PCIe is an interface used to connect high-speed nodes, and while high-speed connection terminals and protocols such as PCIe offer substantial advantages, these types of terminals require high power to operate. For a battery-powered computing architecture like a headset, this would place a huge burden on the system.

In a patent application titled "Error recovery and power management between nodes of an interconnection network", Microsoft describes improving the connection between transmitter nodes and receiver nodes by recovering from error conditions without the need to retransmit data. transmission method to help reduce the system burden on the headset.

In one embodiment, start data streaming. The data stream includes different types of data packets. Error correcting code (ECC) is optionally applied to control data type packets. The transmitter node and the receiver node are connected through hard links with multiple virtual channels. Each virtual channel is associated with a corresponding power consumption node.

When the receiver node receives a control data type packet, error correction is performed without retransmission if necessary. When sending the final data type packet for each virtual channel, the sender node sends the end condition type packet. The corresponding power consumption node corresponding to the corresponding virtual channel transitions from the active state to the low power state.

Certain embodiments are configured to improve power usage of battery-powered devices by identifying when packet transmission across a virtual channel is complete, and by causing the node to enter a low power state when the packet transmission is complete.

An example of an interconnection network 200 is shown here, as shown in Figure 2. Interconnection network 200 may contain any number of interconnection nodes, such as node 205 and node 210. Nodes 205 and 210 can be various types of computing components, such as HPU holographic processing units

As shown in the figure, node 205 is connected to node 210 through high-speed link 215. High speed link 215 may be any type of physical channel connection that supports high speed data transmission.

The described embodiments may virtualize high speed link 215 to include any number of virtual channels. For example, hard link 240 represents high speed link 215. Hard link 240 is shown to include any number of virtual channels, such as virtual channels 245, 250, 255, and 260. Packets are shown traveling through these different virtual channels, such as packet 265.

Each virtual channel is associated with its respective buffer. For example, virtual channel 245 may be associated with buffer 270, and virtual channel 250 may be associated with buffer 275. Other virtual channels are associated with their respective buffers. Each buffer can have a corresponding buffer size of 280. According to the inventive principle, the hardware is capable of maintaining channel bandwidth balance among various requesters (i.e., receiving nodes).

Figures 3 and 4 are examples of head-mounted displays.

High-speed link 410A may be used to transmit high-speed display data and/or sensor data between a computing unit located in the front-end housing and a computing unit located in the rear-end housing. High-speed link 410A is designed to have low latency so that it can pass large numbers of computing cycles or communications. In contrast, low-speed link 405A can be used to provide power, ground or switching mechanisms.

High-speed link 215 in Figure 2 may represent high-speed link 410A in Figure 4, and nodes 205 and 210 in Figure 2 may represent any computing unit shown in Figure 4, such as display circuit 420A, CPU 420B and 440A, GPU 420C and 4406, SOC 420D and HPU 440C.

Figure 8 lists operational requirements 800. Initially, operational requirements 800 include real-time data requirements 805 . As discussed earlier about headsets, the headset used to display content to the user has real-time requirements 805, so it is desirable to avoid retransmitting data

In addition, the content of the headset will dynamically change based on many different factors, including the posture/orientation of the headset, scene changes in the MR scene, interaction with holograms, etc. So when implemented in a headset, data transfer from one node to another in the headset can be required to satisfy real-time data requirements 805 . This means that headset implementations now do not have the ability to retransmit data, as traditional systems do when encountering data errors.

Due to live data request 805, retransmissions to resolve or correct errors are not available. Therefore, Microsoft's invention introduces a new technology to respond to errors.

In addition, operational requirements 800 include low-overhead requirements 810. As mentioned before, hard links between nodes have limited bandwidth. The principle described in the invention provides an error correction technology that not only meets the real-time data requirements 805, but also can place a small amount of extra overhead in the bus protocol.

800 Operational requirements also need to include low power consumption requirements 815. Depending on the specific characteristics of the data flow, embodiments may selectively turn off power consuming nodes to reduce power consumption. For example, in a head-mounted display environment, embodiments may cause the laser to enter a low-power state by detecting specific data packets transmitted between nodes

In order to achieve the above requirements, embodiments cause specific types of data packets to be transmitted from one node to another node in the data flow. FIG. 9 shows an example package 900 that is representative of the package discussed in FIG. 2 . It is worth noting that the data packet 900 can take different forms, including a control data type data packet 905, a non-control data type data packet 910, and an end condition data packet 915.

After all virtual channels have completed sending their respective data streams, the last virtual channel that has completed sending its data stream sends an additional final end condition packet to give the status of all virtual channels. The sending node does not send any new traffic for the virtual channel until all virtual channels have completed transmitting their respective data streams for the scan. When a new scan is initiated or a new frame is initiated, the virtual channel will start sending packets again.

Control data type package 905 is a package type used to control how specific operations are performed. For example, referring to the header display example, the control data type packet 905 may contain information about when the laser is fired, timing information, payload type information, virtual channel information, and may further include information about where the laser pulse is placed.

The control data type packet 905 may also include header information indicating which channel or virtual channel to use. For example, each laser of the headset could be associated with its own virtual channel. Control data type packet 905 may include control information for the corresponding laser of the virtual channel and virtual channel information

Due to the importance of the control data type packet 905, the packet is protected using the error correction code ECC 925. To clarify, since the time and location information contained in the control data type packet 905 is differentially compressed and very important, ECC 925 is used to protect the data from errors that may occur during transmission.

If errors are allowed in the data contained in the control data type packet 905, the entire scan may be corrupted, causing artifacts to be introduced into the resulting image frames.

Figure 10 provides additional description of ECC 1000, which represents ECC 925 of Figure 9. Specifically, ECC 1000 includes at least two types of protection, including single-bit error correction 1005 and double-bit error correction 1010.

In one embodiment, the error protection is x bit error correction and x 1 bit error detection. Select ECC 1000 to ensure ECC 1000 supports error rates. The ECC 1000 is available with 8 bits per control data type packet, or other number of bits selected depending on the length of the control data type packet.

Returning to Figure 9, the non-control data type packet 910 is a packet containing payload information, such as how to illuminate a specific pixel in the scan, such as color intensity, duration, laser pulse size, etc. As such, the non-control data type packet 910 includes pixel data 930 describing the mechanism of how the pixel is illuminated.

Although a single control data type packet can be sent over the virtual channel in each new data flow, any number of non-control data type packets may be sent in the data flow. Additionally, corruption of any single non-control data type packet may result in corruption of only a single pixel.

Headset refresh rates typically range from 90 Hz to 120 Hz, and the resolution is very high, so users may not notice the damage to individual pixels. Given these conditions, embodiments selectively avoid imposing ECC on non-control data type packets 910. Errors that occur in non-control data type packet 910 can be simply accepted because the impact of these errors is minimal

By applying ECC 925 to the control data type packet 905, since ECC 925 allows error correction and detection at the receiving node, there is no need to retransmit the control data type packet 905, and real-time processing can be performed, thereby satisfying the requirements in Figure 8 Real-time data requirements 805.

By applying ECC 925 only to control data type packets 905 and not to non-control data type packets 910, embodiments meet the low overhead requirement 810 because the additional data only applies to a single packet rather than multiple data pack. Therefore, the additional overhead is also negligible

Additionally, by applying ECC, embodiments are able to automatically recover when an error is detected, thus meeting the automatic hardware recovery requirement 825. Therefore, embodiments impose different data protection requirements based on data type or rather different data packet types.

End condition package 915 is a package used to trigger when the data flow is completed. In other words, when all non-control data type packets have been sent, the end condition packet 915 will be sent as the last packet of the data flow

If no pixels are illuminated by a specific laser in a specific scan, there may be no control data type packets or non-control data type packets transmitted on the laser's corresponding virtual channel. Instead, only end condition packet 915 can be sent. To help mitigate possible errors, multiple end condition packets can be transmitted.

Figure 11 shows an example scenario involving multiple virtual channels 1100 and the three different packet types introduced in Figure 10 are being transmitted. Figure 11 shows a scenario with 12 virtual channels and data flow flowing from left to right. Additionally, different bag types are illustrated using different shading techniques. In this example scenario, each virtual channel corresponds to a single laser in the headset, and each virtual channel has its own corresponding data stream being transmitted.

Figure 11 shows how the data flow for Channel 1 starts with a control data type packet, then includes five non-control data type packets, and then includes an end condition packet. Channel 2 is similar, it includes a control data type packet, two non-control data type packets and an end condition packet. Channel 1 and Channel 2 are transmitting data because the laser corresponding to the channel will be used to illuminate the pixels in the scan.

Channel 3, on the other hand, corresponds to the laser that will not be used to illuminate the scanned pixels. Therefore, the data stream transmitted through channel 3 only includes an end condition packet. This end condition packet usefully informs that the corresponding laser will not be used for this particular scan.

Accordingly, if the laser is not intended to be used during a specific scan, the data stream transmitted through the laser's virtual channel will only include one or more end condition packets

Figure 12 focuses on virtual channel 1200, which represents virtual channel 1100 in Figure 11. Figure 12 shows how the end condition packet is transmitted over channel 1. The end condition packet operates as a trigger 1210 to notify the system that the corresponding power consumption node can transition from the power consumption mode to the reduced power consumption mode. This triggering is beneficial because the system now knows that the power node is idle and can transition to a reduced power mode.

For example, in a head-mounted display environment, the head-mounted display includes a laser transmitter 1215, which itself includes lasers 1215A, 1215B, and 1215C. Channel 1 is a virtual channel specifically corresponding to laser 1215A. Transmitting a terminal condition packet over Channel 1 causes the system to recognize that use of laser 1215A is now complete and therefore can transition from active mode 1220 of generating laser light to low power mode 1225 or possibly even off mode 1230.

As shown in Figure 12, the end condition has been triggered, causing laser 1215A to stop emitting laser light, while lasers 1215B and 1215C may still continue to emit laser light. According to Figure 11, by transmitting the end status data packet in each different virtual channel, the node corresponding to each virtual channel enters a low power consumption state, thereby maintaining the system's power

Simply turning back to Figure 11, channel 1 is shown as the last channel transmitting the end condition packet. No matter which virtual channel transmits the end condition data packet last, the task of the data flow of the virtual channel is to perform the so-called encapsulation operation, in which the end condition data packet constitutes the "final" end condition type data packet 1315, as shown in Figure 13

The final end condition type packet 1315 provides additional information to the receiving node indicating that all data flows for all other virtual channels have been completed. This functionality is beneficial in a head-mounted display environment because the final End Condition Type Packet 1315 signals the end of the scan.

Related Patents: Microsoft Patent | Error recovery and power management between nodes of an interconnection network

Microsoft patent application titled "Inter-node Error Recovery and Power Management" was originally filed in August 2023 and recently published by the U.S. Patent and Trademark Office

It should be noted that, generally speaking, after a U.S. patent application is reviewed, it will be automatically published 18 months from the filing date or priority date, or it will be published within 18 months from the filing date at the request of the applicant. Note that publication of a patent application does not mean that the patent is approved. After a patent application is filed, the USPTO requires actual review, which can take anywhere from 1 to 3 years.

In addition, this is just a patent application, there is no guarantee that it will be approved, and it cannot be determined whether it will be truly commercialized and produce practical application effects

The above is the detailed content of Improving HoloLens internal PCIe data link transmission: Microsoft shares AR/VR patents. For more information, please follow other related articles on the PHP Chinese website!