(even with a Chinese test)!
FEATURED PAPER
By Federico Minelle, Franco Stolfi, Roberto Di Gioacchino
Rome, Italy
Abstract
The article shows the one-year-long benchmarking results, which suggest a promising future for AI-assisted project management. For sure, but not only from USA: please try a Chinese support, too…
The quality test reveals clear signs of improvement and a strong reduction in some critical issues, especially in terms of reliability. It seems that our AI champions have done their homework!
Anyway, in the world of AI, a year is practically a geological era! Maybe the next project manager could be an algorithm with a good sense of humor, too!
Brief History: When AI Decided to apply as a Project Manager assistant
About a year ago, while generative Artificial Intelligence (gen-AI) tools continued their unstoppable march towards conquering the world (or at least our smartphones), some brave researchers decided to benchmark them in an unexpected field: project management [1, 2], as assistant to a project manager of the Italian Public Administration (PA). Who would have ever thought that one day we would be evaluating algorithms to manage a project of the Italian PA?
In the meantime, international project management gurus have started to proclaim out loud that gen-AI (and in the future also its more advanced cousin, AGI-Artificial General Intelligence) will revolutionize the work of project managers [3, 4, 5, 6]. Certainly: the future of project management looks to be interesting, unpredictable and, hopefully, with fewer useless meetings. After all, if there is anyone who can appreciate the efficiency of an AI support, it would be a professional project manager!
Benchmark current plan
More seriously, the previously adopted approach [1] was confirmed, which was applied to each of the selected USA champions (ChatGPT, Claude and Gemini), and later to the Chinese new entrant (Deepseek):
- assessment plan set up consistent with the “Integrated Project Management Practices” scheme, according to the ISO 21502 standard;
- “sample project” is a simplified version from an actual ICT project of the Italian PA, prompted at the beginning of the conversation;
- quality tested by submitting the same questions to each tool, about the management of the sample project;
- any answer was evaluated using the same already applied quality model (Table 1), by assigning a rating to every sub-characteristic on a 0-3 scale (in ascending order): threshold level was set at 2 (partially satisfactory);
- benchmarks for the USA champions were both “transversal” (comparison of the quality characteristics among the selected tools, in approximately the same time period) as well as “longitudinal” (comparison of the quality of the same tool, along the time), starting in Spring/Summer 2023 and afterwards.
- Deepseek test was performed in Winter 2025, just after its availability to abroad users[1].
Table 1 – Quality Characteristics (ChatGPT) – Personal assistant
to the Project Manager of the ISPA project
Benchmark Results: competitive features (USA champions)
In the Summer 2024 benchmark (for the USA champions only), our digital competitors shined in communication (compared to previous editions), improving their “usability” scores. It’s as if they’d all taken a crash course in algorithmic public speaking!
The fluidity of conversations led also to an overall improvement in the “suitability” of content, without sacrificing “usability”. It’s as if our digital friends had learned to exploit the top of project management: precise and without inserting too many unreliable references in their answers!
The winner of this competitive edition (by a hair, we would say in horse racing) was Claude, who earned a virtual gold medal for his ability to reference many sources and provide real links (Figure 1). It’s like having a hyperactive librarian, mentioning proper insights, referred from well-recognized project management books!
Good news on the hallucination front: they seem to have disappeared! Our chatbots have stopped inventing fancy links and now prefer to give us instructions on how to search for information on the web. It’s a bit like if your personal assistant, instead of making up stories, told you: “Look, search for yourself, it’s better!”
More…
To read entire paper, click here
How to cite this work: Minelle, F., Stolfi, F., Di Gioacchino, R. (2025). AI to support PM: a one-year-long benchmark (even with a Chinese test)! PM World Journal, Vol. XIV, Issue III, March. Available online at http://pmworldlibrary.net/wp-content/uploads/2025/03/pmwj150-Mar2025-Minelle-Stolfi-DiGioacchino-AI-to-support-PM.pdf
About the Authors
Federico Minelle
Rome, Italy
Federico Minelle graduated in Physics at “Sapienza” – Rome University, discussing his Master’s degree thesis on an Artificial Intelligence (AI) research project. Afterwards, for more than 40 years he consulted in Business Organization, Information Systems and Project Management.
As a partner at Accenture and then as senior partner of the Italian consultancy firm PRS – Planning, Ricerche e Studi, he managed and monitored significant projects in several industries, mainly in Engineering and Construction (e.g. manufacturing and nuclear power plants, transportation infrastructures) and in Government (ICT and process innovation).
Federico taught for more than 20 years Business Information Systems in the Computer Science Dept. of “Sapienza” – Rome University, where he also taught Project Management & ICT for the academic master’s degree in ICT Governance and Audit. At the same time, he often lectured about similar topics to Government officers attending SNA (National Public Administration School) courses.
He was Editorial Director and then Scientific Director of the Italian journal “Il Project Manager”, developing a fruitful cooperation with the PM World Journal. Just after retiring at mid 2022, he was enrolled in the Scientific Council of the same Italian journal, where he continues to write editorial notes, papers and book reviews.
Since 2018 Federico Minelle is an International Editorial Advisor for the PM World Journal (PMWJ) and by extension for the PM World Library.
He is an honorary fellow of ISIPM (Italian Institute for Project Management).
Federico has authored several papers and seminars on Cost/Benefit analysis for ICT Government projects and on PMO functions in ICT organizations. Federico can be contacted at minelle@di.uniroma1.it.
Franco Stolfi
Rome, Italy
Franco Stolfi graduated in Computer Science at Salerno (Italy) University. For more than 40 years he has been consulting in Business Organization, Information Systems and ICT governance (Project, Security, Service and Quality Management).
Previously as director of an ICT SME company and now as senior partner of P.R.S. Planning, Ricerche e Studi (an Italy based consulting firm on ICT governance, security, monitoring and training), he manages PMO services and monitors large ICT projects on process innovation, digital transformation and ML/Blockchain, for Government Departments and Companies.
He was contract professor on Business Information Systems in the Computer Science Dept of “Sapienza” – Rome University, he also taught Project Management & ICT Governance for the academic master’s degree in ICT Governance and Audit and he held several seminars to Government officials.
Member of the editorial staff of the “Il Project Manager” Italian journal, where he wrote several articles on project management. He also co-authored many papers & studies (e.g.: Quality in the websites of the Italian Public Administration; Quality ICT procurement guidelines issued by the Government Agency for ICT; Project Life Cycle Economics book published by Gower).
He is certified Prince2 V2 practitioner, Lead auditor for IT service management ISO-IEC 20000, ITIL, Lead auditor for quality management systems ISO/9001, Lead auditor for Information Security Management Systems ISO/27001, ICT Project Manager certified by Accredia and ISIPM-Prado© Auditor.
He is board member of ISIPM Professioni (Italian Institute for Project Management – professional Project Manager).
Franco can be contacted at stolfi@prsmonitor.it
Roberto Di Gioacchino
Rome, Italy
Graduated in Electronic Engineering at “Sapienza” – Rome University, presenting a Master’s degree thesis on Project Management.
For more than 30 years he’ has been consulting in Business Organization, Information Systems and Project Management.
He is senior partner and administrative director of P.R.S. Planning, Ricerche e Studi (an Italy based consulting firm on ICT governance, security, monitoring and training), he manages PMO services and monitors large ICT projects on process innovation, digital transformation and ML/Blockchain, for Government Departments and Companies.
Previously member of the Executive Board of ASSINTEL (Italian Association of ICT Companies)
He is certified: ITIL V3, Prince2 Practitioner, ISIPM-Prado© Auditor
Roberto can be contacted at: digioacchino@prsmonitor.it
[1] See in appendix a short sample of relevant Q/As (Screenshot 1, Screenshot 2, Screenshot 3)
