patents

2025

1. Patent Application

   Systems and methods for static cached decoding

   Shaojie Zhuo, Ramchalam Kinattinkara Ramakrishnan, Xiaopeng Zhang, and 4 more authors

   2025

   Pending Abs Bib PDF

   Cached decoding systems and techniques are described. A system (e.g., decoder) receives an input token (e.g., input vector). The system applies a projection tensor (e.g., a projection matrix) to the input token to generate a feature tensor (e.g., a key tensor or a value tensor). The system processes at least the feature tensor and at least one previous feature tensor using at least one attention calculation to generate an output token. The at least one previous feature tensor is retrieved from a buffer. The at least one previous feature tensor can be stored in the buffer after having been previously calculated based on application of the projection tensor to a previous input token (e.g., from a previous iteration before the iteration in which the input token is received).

   @patent{scd,
     author = {Zhuo, Shaojie and Kinattinkara Ramakrishnan, Ramchalam and Zhang, Xiaopeng and Lin, Yicheng and SU, Chenzheng and Shen, Liang and adams},
     title = {Systems and methods for static cached decoding},
     year = {2025},
     assignee = {Qualcomm Incorporated},
     status = {Pending}
   }

2024

1. Patent Application

   Selective neural network pruning by masking filters using scaling factors

   Ramchalam Kinattinkara Ramakrishnan, Vahid Partovi Nia, and Eyyüb Sari

   2024

   Granted Abs Bib PDF

   A method and system for pruning a neural network (NN) block of a neural network during training, wherein the NN block comprises: a convolution operation configured to convolve an input feature map with a plurality of filters, each filter including a plurality of weights, to generate a plurality of filter outputs each corresponding to a respective filter; an activation operation configured to generate, for each of the filter outputs, a respective non-linearized output; a scaling operation configured to scale the non-linearized output generated in respect of each filter by multiplying the non-linearized output with a mask function and a respective scaling factor that corresponds to the filter. During training: for each scaling factor corresponding to a filter, learning the scaling factor by minimizing loss of a loss function including a first regularization function with respect to the scaling factor; and if a value of the scaling factor satisfies a predetermined criterion, selectively pruning the filter corresponding to the scaling factor by masking the filter from the convolution operation.

   @patent{dmp,
     author = {Kinattinkara Ramakrishnan, Ramchalam and Partovi Nia, Vahid and Sari, Eyyüb},
     title = {Selective neural network pruning by masking filters using scaling factors},
     year = {2024},
     assignee = {Huawei Technologies Co., Ltd.},
     status = {Granted}
   }

2. Patent Application

   Single search for architectures on embedded devices

   Chen Feng, Xiaopeng Zhang, Shaojie Zhuo, and 5 more authors

   2024

   Pending Abs Bib PDF

   A processor-implemented method for a neural architecture search (NAS) starts by generating an over-parameterized super network having multiple layers. The super network has multiple operator types. Each of the layers includes a largest super kernel corresponding to a search space. The method also includes performing gradient descent to evolve a largest super kernel to a small kernel corresponding to the search space in order to generate a range of kernel encodings. The method further includes identifying a subset of kernel encodings from the range of kernel encodings, for each layer of the super network, based on the gradient descent. The method determines a set of candidate architectures based on the subset of kernel encodings, each of the candidate architectures having a different model size. The method selects a target model, from the set of architectures, based on meeting hardware specifications, and then applies the target model.

   @patent{nas,
     author = {Feng, Chen and Zhang, Xiaopeng and Zhuo, Shaojie and Kinattinkara Ramakrishnan, Ramchalam and Su, Chenzheng and Shen, Liang and Han, Zi Wen and Lin, Yicheng},
     title = {Single search for architectures on embedded devices},
     year = {2024},
     assignee = {Qualcomm Incorporated},
     status = {Pending}
   }