Communal AI – Open, Collaborative & Accessible LLMs

Abstract: Developing better Language Models would benefit a myriad of communities. However, it is prohibitively costly. The talk would describe collaborative approaches to pretraining, such as model merging, which allows the combining of several specialized models into one. Then, it would introduce efficient evaluation to reduce overheads and touch on other accessible and collaborative aspects that best harness the expertise and diversity in Academia.