How AI Impacts Skill Formation

How AI Impacts Skill Formation

Research questions. The paper asks whether AI assistance improves productivity when workers must learn a new skill, and whether that assistance harms or helps skill formation. It focuses on novice developers learning a new Python asynchronous programming library and whether AI use creates a tradeoff between short-term completion and longer-term competence.  

Methodology. The authors run randomized experiments where participants complete coding tasks using the Trio Python library, either with or without an AI assistant based on GPT-4o. They then assess learning through a 14-question evaluation covering conceptual understanding, code reading, and debugging, supplemented by qualitative analysis of screen recordings and AI-use patterns.  

Findings. AI assistance reduced evaluation scores by 17 percent, or about two grade points, while not producing a statistically significant improvement in completion time on average. The authors identify six AI interaction patterns and find that cognitively engaged uses, such as asking for explanations or conceptual help, preserve learning better than full delegation.  

Why it matters. The paper is important because it challenges the idea that AI-assisted productivity automatically builds competence. For AI supervision and alignment work, it suggests that relying on AI can weaken the very skills people need to evaluate, debug, and oversee AI-generated work.