Sitemap
A list of all the posts and pages found on the site. For you robots out there is an XML version available for digesting as well.
Pages
Posts
Self Distillation
Published:
Self Distillation has recently come up as a promising direction for language model post training. It targets two major shortcomings of dominant RL based post training algorithms like GRPO. GRPO requires a verifier signal (ex : answer correctness) which might be hard to obtain for tasks where there isnt a clear notion of correctness. GRPO also has a credit assignment problem : for positive advantages, every token in the rollout is equally upweighted. The algorithm cannot assign granular credit to parts of the rollout, unlike its predecessor PPO, which suffers from its own inefficiencies and instabilities.
Flow Matching
Published:
Flow matching gives us a way to model complex real world distributions (target) from simpler or known distrubitions (source).
portfolio
Portfolio item number 1
Short description of portfolio item number 1
Portfolio item number 2
Short description of portfolio item number 2
publications
Zero-shot Cross-lingual Transfer With Learned Projections Using Unlabeled Target-Language Data
Published in Association for Computational Linguistics, 2023
An adapter based framework to improve cross lingual transfer when target languages are known at training time.
AdaKD: Dynamic Knowledge Distillation of ASR models using Adaptive Loss Weighting
Published in arxiv, 2024
A curriculum learning based distillation method for transformer based speech recognition models.
talks
Talk 1 on Relevant Topic in Your Field
Published:
This is a description of your talk, which is a markdown files that can be all markdown-ified like any other post. Yay markdown!
Conference Proceeding talk 3 on Relevant Topic in Your Field
Published:
This is a description of your conference proceedings talk, note the different field in type. You can put anything in this field.
teaching
Teaching experience 1
Undergraduate course, University 1, Department, 2014
This is a description of a teaching experience. You can use markdown like any other post.
Teaching experience 2
Workshop, University 1, Department, 2015
This is a description of a teaching experience. You can use markdown like any other post.