Post-doc : « Towards sustainable AI: Measuring the Computational Cost of Deep Learning » – 18-month contract

Working environment:

The Mines-Télécom Institute (IMT) is the 1st public group of Grandes Écoles (schools of engineering and management) in France. Consisting of eight public schools and two affiliate schools, IMT builds a rich ecosystem of academic and industrial partnerships dedicated to higher education, research and economic development.

IMT gives a central role to digital sovereignty and sobriety in its strategy, realized via activities of its Data&AI scientific community targeting sustainable AI, among others. This strategic axis led IMT to join the excellence network ENFIELD, which structures the research of thirty European partners in the context of trustworthy and green AI (European Lighthouse to Manifest Trustworthy and Green AI, voir https://www.enfield-project.eu/). Mines Saint-Étienne, one of the eight Grandes Écoles of IMT, is opening a post-doc position to strenghen its presence in ENFIELD.

Mines Saint-Étienne conducts research on sustainable AI from the angle of computational cost of machine learning and lifecycle assessment of AI systems.

Scientific challenges:

Language models and other foundational models based on deep neural networks have become ubiquitous in a few years. However, already after the introduction of the Transformer architecture in the literature was the question of their computational cost raised. Their great generalization abilities mainly comes from the fact they possess a large number of configurable parameters, compared to their predecessors. Yet, one can empirically observe that progress made since 2012 in machine learn- ing has come with an exponential growth in the computational cost to train a model.

The growing adoption of neural networks in information systems raises the question of the sustainability of AI with respect to our environment. In turn, controlling the computational cost of AI calls for solid indicators to make decisions prior to the design phase of any information system. Software eco- design e.g. imposes to justify that any solution with high computational/environmental cost fulfils a specific need and cannot be replaced with a lower-cost solution.

Yet, currently, there is no a priori indicator related to the computational cost of a machine learning model. Numerous dimensions come into play, including the amount of training data, the complexity of the model or hardware performances. Some of these dimensions are available online: Hugging Face, as an example, publishes model cards that indicate how many trainable parameters are model has, what training data has been used and, sometimes, an estimate of energy consumption during training. However, a model card provides no information after the model is trained (i.e. at inference time).

Current practices in AI call for the development of an indicator of computational cost that is suitable for eco-design. This indicator could be evaluated on the various models developed by ENFIELD partners, as well as members of IMT’s Data&AI scientific community.

Main missions:

The main mission of the post-doc will be to design a static analysis approach to estimate the computational cost of a machine learning model implemented with a tensor library such as PyTorch, Tensor- Flow or NumPy. The approach will have to take several dimensions into account, including data size, model size and hardware characteristics such as the size of memory and number of parallel cores.

The post-doc will also have the mission to moderate scientific exchanges between ENFIELD partners, structured in a matrix made of pillars (Green AI, Adaptive AI, Human-Centric AI et Trustworthy AI) and application domains (Energy, Healthcare, Manufacturing et Space). Each pillar and each application domain is associated with a working group that has to coordinate with other working groups.

Main activities :

The post-doc includes the following activities:

• Review of available models on Hugging Face, of models developed at IMT and of models developed by ENFIELD partners,
• Design of an indicator of computational cost for these models (e.g. a number of FLOPs),
• Development of a static analysis tool to calculate estimates from code (e.g. from PyTorch functions)
• Participation in ENFIELD working group
• Participation in activities organized by IMT’s Data&AI scientific community

Profile:

Skills, knowledge :

• PhD in AI, applied mathematics or software engineering

Working conditions:

• 18-month public contract (standardized salary based on experience levels)
• Beginning around June-September 2024
• Full-time
• Based in Saint-Étienne
• 49 days of vacation
• Partial remote work allowed

Application modalities:

Applications (CV, motivation letters) are to be submitted via RECRUITEE by May 09th, 2024 :

https://institutminestelecom.recruitee.com/o/post-doctorante-ou-post-doctorant-pour-une-ia-sou-tenable-etude-du-cout-calcul-de-lapprentissage-profond-cdd-de-18-mois-2

For more information:

Victor Charpenay (associate professor), victor.charpenay@emse.fr

Olivier Boissier (head of Institut Fayol), olivier.boissier@emse.fr

Administrative staff:

Julie Jaffre, julie.jaffre@emse.fr, 04 77 42 00 17