Dates: 2024
Principal Investigator: Dr. Irfan Ahmad
Description: This project aims to investigate various aspects of prompt engineering and machine translation (MT) for improving instruction-following models, particularly in Arabic. The study will begin with a thorough review of the literature and state-of-the-art methods to design prompt templates and explore questions such as the effectiveness of single versus diverse templates and the impact of prompt chaining and other ideas such as CoT, self-ask, and ART. Instruction samples will be collected from different domains, including Humanities, Social Sciences, and STEM, from platforms like Stack Exchange and Wikipedia. The collected samples will undergo cleaning and restructuring, if necessary, to study the influence of templates on model effectiveness. Additionally, manual prompts authored by domain experts will be included. The research will also explore the design of prompt chains for multi-turn dialogues and assess the effectiveness of MT for instruction tuning in Arabic, comparing it to original Arabic prompts and studying its role in Arabic natural language processing (NLP). The project will compare its approach with existing state-of-the-art models and evaluate the performance of instruction-tuning models with varying sample sizes and templates. Evaluation metrics will include the quality of generated responses, analysis of response effectiveness across different types of questions, and the impact of prompt quality, diversity, and structure on response quality. A tool suite to support the process will be developed.