Redefining Computer-Aided Tuning (CAT) with Reinforcement Learning
Designing a high-performance microwave filter can be a daunting task for RF engineers. The process involves analyzing stringent requirements such as center frequency, bandwidth, return loss, insertion loss, and more, which can become increasingly complex with higher-order filters or complex designs.
The Problem
During the designing phase, a RF engineer spent most of the time just in tuning the structural parameters to meet the correct filter’s responses in simulations. Additionally, conventional methods of filter design rely on mathematical approaches such as convex optimization, which can be time-consuming and require robust engineering to achieve a fully convex objective function.
This objective function is used to describe how far or close is the current filter’s response to the desired response with a scalar value. Usually, this scalar value represented as loss, and the objective of the convex optimization is to perturb a set of structural parameters such that this loss is minimized, thus obtaining the desired filter’s response.
To optimize and tune the filter design once all sensitive structural parameters had been identified, engineers typically rely on simulation software such as Electronic Design Automation (EDA) that can support tuning and optimization techniques. However, even with EDA software, the design process can still be incredibly time-consuming, particularly for 3D designs that require extensive simulations and iterations to compute responses accurately.
Imagine one simulation of a 3D structure took around 4 minutes to complete. An ideal session of optimization process would take around 100 iterations. This means more than 6 hours spent to just tune a filter! And that is without addressing the underlying uncertainty such as premature convergence to local minima or if the loss is still big.
The Solution
This is where Reinforcement Learning (RL) has come to grab our attention. To address these limitations, we are turning to RL-based CAT, which utilizes AI methods to automate the optimization and tuning process. Thanks to the recent achievements in the field of RL, this AI has emerged as a key technology in industrial applications such as robotics, Unmanned Autonomous Vehicles (UAV), OpenAI’s ChatGPT and DeepMind’s Alpha GO.
The reason why RL is more suitable for tuning and optimization problem, other than other AI machine learning architecture, is mainly because it is feedback-driven, and it learns by continuously updating its policy from many self-plays (trials and errors) environment.
This policy is basically the “experience” of what actions to take given a state. In CAT context, actions refer to the perturbation of structural parameters, and the state (can be a vector or matrix) describes the distance between the current response to the desired one. By mapping the nonlinear relationship between the filter’s structural parameters and its response, this advanced technique can quickly provide the optimal set of structural parameters based on the desired filter response.
The Performance
Here, we demonstrate the capability of RL in tuning some 3D structures, an 4th-order Chebyshev bandpass waveguide and an Extracted Pole Unit (EPU).
High level workflows are drafted as shown:
In figure 1, an 4th-order waveguide BPF 3D design with all resonators’ length, L are tuned with RL by evaluating its S-parameters. For simplicity, the objective of RL to learn is to maximize the policy such that output actions could perturb the set of parameters so the S-parameters are closely matched to the desired S-parameters.
In figure 2, RL can tune EPU structure by evaluating group delay as the filter’s response. Like the first sample, the objective is to search optimal set of parameters to obtain good agreement to the desired response.
Their results and tuning process can be visualized as follow:
RL is trained in a self-plays environment; its policy is updated through trials and errors. The trained model can be confidently deployed to tune highly detuned filters. By continuously feedback the model with current evaluations of filter’s responses, RL could know which direction to tune on each parameters, and what deviations to take. Gradually, the filter’s response will come close to the desired one with lesser steps taken to complete the optimization process when compared to conventional methods.
Better and Faster
With Reinforcement Learning-based CAT, engineers can now design and fabricate high-performance microwave filters more efficiently and accurately than ever before. This cutting-edge approach not only saves valuable time and resources, but it also enables RF engineers to focus on higher-level tasks while achieving optimal system performance. From a business point of view, this will increase the time-to-market of products, reducing cost and allowing larger operations.
FILPAL has achieved the capability to incorporate Reinforcement Learning into enhancing our EDS HF. Find out more about our EDS HF and the AI we have here: https://www.filpal.com/eds-hf-ai
Originally published at http://filpal.wordpress.com on March 12, 2023.
