Replace Game Property with Variable
When using python in the game engine you can use a script or a module. While a script will be run from start to end, a module is a collection of functions that can be called indiviually, the "global" code in a module is run once when the module is first activated, which makes it a good place for a one off initialisation that you are looking for.

import bge

GameScore = 50
print('init once here')

def ChangeScore(cont):
    global GameScore
    ksens = cont.sensors["Keyboard"]
    sce = bge.logic.getCurrentScene()
    scoreDisplay = sce.objects['ScoreDisplay']

    for key,status in ksens.events:
        if status == bge.logic.KX_INPUT_JUST_ACTIVATED:
            if key == bge.events.UPARROWKEY:
                GameScore += 5
            if key == bge.events.DOWNARROWKEY:
                GameScore -= 5

            scoreDisplay['Text'] = str(GameScore)


Paste this into a textblock and end the blocks name with .py you then use a python controller, set to module and enter the textblock name without the py extension followed by the function name like sample.ChangeScore. You will see the "init once here" will only get printed once when the module is first run.