Fix in scorer manager in picking the best target

[mayabar]

No description provided.

[vMaroon]

isFirst can be completely dropped, because it doesn't fix the negative score case, since the array is not sorted (which would beat the point). If you want to account for negative scores, maxScore should be set to math.SmallestNonzeroFloat64.

The code block can look like:

	// select pod with maximum score, if more than one with the max score - use random pods from the list
	var highestScoreTargets []*types.PodMetrics
	// score weights could be negative
	maxScore := 0.0

	for pod, score := range podsTotalScore {
		if score > maxScore {
			maxScore = score
			highestScoreTargets = []*types.PodMetrics{pod}
		} else if score == maxScore {
			highestScoreTargets = append(highestScoreTargets, pod)
		}
	}

I think negative scores should be prohibited textually through the Scorer interface (to avoid checking it), because allowing them would let such scorers to invade and overwrite the scoring of others. For the same reasons, scores should be normalized (cc @elevran @nirrozenbaum).

[nirrozenbaum]

I agree with @vMaroon. scorers should never return a negative value by definition.
the "normalized score" of a score should reflect in percentage the recommendation of the scorer for the pod (e.g., a float with value within the range 0-1).

[nirrozenbaum]

I recommend reading the NormalizeScores section in kubernetes scheduling-framework:
https://kubernetes.io/docs/concepts/scheduling-eviction/scheduling-framework/

[elevran]

Negative is useful (e.g., prefer the pod with the best cache coverage and lowest load can be expressed naively as kv - load when both are normalized). However it is better to use weights for that and not in the score.
The downside is that we need to decide what the value represents. Working on the assumption that we prefer Pods with higher score would mean that the LoadScorer would rate pods as "1 - load" which would give lower score to loaded Pods. This would not align with the negative weights.