feat(skore): Registry for custom and extra metrics

[divakaivan]

Is your feature request related to a problem? Please describe.

Right now skore provides some default metrics available in a report with report.metrics.<metric_name> and also report.metrics.summarize().frame(). It'd be nice if a user could also save any extra metric they want their report to have and the report exposes access to. If I I have a custom_metric, in order to see it's results I need to do

report.metrics.summarize(
    scoring=metric_function,
    scoring_kwargs={"response_method": "predict"},
)

or

report.metrics.custom_metric(
    metric_function=metric_function,
    response_method="predict",
)

which requires me to have access to the metric_function function's definition.

In addition, if I want to see report.metrics.summarize().frame(), I cannot see the default metrics + my metric by default. I'd need to pass all the metrics I want to summarize(scoring=["rmse", "r2", metric_function]). I think it would be nice if I could save a custom metric and whenever I do report.metrics.summarize().frame() all the metrics (the default ones by skore and mine) are shown. And also, if I share reports, I wouldn't necessarily expect the other user to have access to my metric_function, so if I could save my metric in some way into the report that'd be nice.

Describe the solution you'd like

If I want to save a custom metric to be part of my report, I'd like to be able to do something like

report.metrics.save_custom_metric(name="my own metric", metric_function=metric_function)

Then I (or if I share the report, a colleague) can do this to get the output:

report.metrics.custom_metric("my_own_metric")

Also if I have a custom metric saved, I would expect to see it alongisde the provided by default metrics. (and maybe have a flag like summarize(include_custom=True)

report.metrics.summarize().frame()
# RMSE ...
# R2 ...
# my_own_metric

Describe alternatives you've considered, if relevant

No response

Additional context

inspired by a comment on discord by pyphi

[glemaitre]

I think it is a good idea. We previously discussed IRL about creating a sort of register for metric such that you can plug new metric and it should be used then by default. It is really aligned with what you propose here.