(Full-time) May 2023 - Aug 2023, (Part-time) Aug 2023 - Dec 2023
Research Scientist/Engineer Intern
• Achieved an increase of 4.74 units for Rouge score and 3.60% for Accuracy@1 improvements for few‑shot learning in Flan‑T5 transformer, by expanding their capacity to be able to process more in‑context example within the same context window length through sub‑batching. • Inched closer to finetuning‑like performance through pure in‑context learning (ICL) by 2.16 units of Rouge score and 3% for Accuracy@1 through mesa‑optimization where the transformer acts like an optimizer itself during inference. • Improved the cross‑domain transfer capabilities of a transformer (Flan‑T5) by 1.68 units for Rouge score and 1.3% for Accuracy@1 through incorporating both cross‑ and within domain question‑answer samples within a limited context window length of 512 tokens. • Evaluated and verified the effectiveness of both sub‑batched ICL and mesa‑optimization during inference on both Adobe and public datasets.