The Berlin Network of Labor Market Research (BeNA) is pleased to announce its upcoming Skills Camp on Classification, Performance and Benchmarking with LLMs with Martina Pocchiari (ESADE). The workshop will take place on December 7–8 at DIW Berlin, and registration is now open.
The workshop provides an overview of the following topics:
- LLM APIs and programming interfaces for working with them
- Recent models from different providers and prompting guides
- Benchmarking classification outputs against other LLM models
- Benchmarking classification outputs against human responses
- Reporting and replicability of results, with insights from the peer review process
Live coding, either on participants’ computers or through a Jupyter notebook, will allow participants to replicate a classification and reporting infrastructure for their own research.
Event details
- Dates: December 7–8
- Exact times: TBA
- Location: DIW Berlin
- Lecturer: Martina Pocchiari (ESADE)
Preparation
There are no reading requirements. Participants who would like to read more can consult this discussion article on the pros and cons of using generative AI for classification tasks: https://journals.sagepub.com/doi/10.1177/00222429251349882
The workshop will use Python, Visual Studio Code, and its Python extensions. Setup guidance is available in the Python documentation for Visual Studio Code and the getting started tutorial.
We recommend that participants prepare a small dataset, ideally from their own research, with text variables they can classify using the methods learned in class. They should also prepare a construct they wish to classify, grounded in the measurement literature. A dataset and construct will be provided for participants who do not have them.
Registration
In case of oversubscription, BeNA members and PhD students will be given priority.