What is the minimum amount of sugar needed for getting the desired texture in this banana marmalade?
There is no way to predict the amount you want. If you want to know it, the only way is extensive experimentation. 

You observed two outcomes, which I will call "amber" and "boiled" for short. If I got you right, you assume that the difference between them is determined by the amount of sugar, and are asking about the minimal amount of sugar to reach the "amber" state. 

Preparation: isolate the responsible factor

You admit that the amount of sugar was not the single factor you varied. You also left out the flaxseeds, which is a major factor in texture. So, before you start the experiment proper, you will have to prove that it was not the flaxseeds, by making the 130g version with them. If you get the amber version, the experiment can continue. If you don't, the whole idea is moot anyway, because it was based on a false assumption. You could set up a similar experiment without flaxseed then - or simply make the recipe without flaxseed and with zero sugar, to see if it doesn't provide the effect you wanted all by itself. 

The theory behind the experiment

So, assuming that you have proven that it is not the flaxseeds. Then you have two possible cases. Either this is a linear system, or a nonlinear one. In a linear system, there are not really two states of "amber" and "boiled", but a long, smooth spectrum, of which you observed two (or actually three) points. In that case, versions made with a middle amount of sugar will have characteristics somewhere between "amber" and "boiled", it is just your human brain with its penchant for dividing the world that assigns some of them into the "boiled" and others to the "amber" category. If that's the case, then the question of the boundary between "amber" and "boiled" is subjective. The only way to know where you personally will assign that boundary is to continue the experiment by making many more batches with different amounts of sugar and find out where your personal minimum lies. 

The other interesting possibility is that of a nonlinear system, where there is no spectrum, but the end state is always objectively either "amber" or "boiled" and there is a tipping point in the amount of sugar which makes the whole system fall exactly into one of those states, without an in-between state. If this is it, you will never be able to mathematically model the process which creates the two states - humanity does not yet have the mathematics for that, and there are signs it may never have them. If this is the case, then there should be a wide swath of parameter space (= sugar amount) that practically always produces the "amber" state, another wide swath that produces "boiled", and a border region between them of unknown width that produces either the one or the other, without a way to predict which one will be. The only way to find out the minimum amount of sugar to produce the amber state, as opposed to being in the border state, will require repeated series of experiments. 

The experiment itself

We don't know which case we have, but luckily, the experimental protocol for both will be the same. First, you should do a simple binary search to quickly find something that looks like a border region. This means: make and record batches of 130 and 70 g (the best thing would be to run the experiments in one day and keep samples, since you probably cannot record all relevant parameters in a home kitchen, and keeping the samples will make them change with time), then search in the middle (maybe that's why you tried 100 g already), then the middle between the two changing states, etc. After you have ran that, you will also have a good idea if you are dealing with case 1 or 2 (unless your search converges too quickly). Once you have it, you should enter the second phase, where you randomly choose 5-10 amounts of sugar close to the turning amount indentified in the binary search, and make those. The reason behind it is that your border will have some width in both cases, in the linear case because of subjectivity (if you are faced with something in the middle between amber and boiled, on some occasions you will perceive it as belonging to the amber category, on some as belonging to the boiled category) and in the nonlinear case because the border is not one-dimensional, but fractal-dimensional, so it has some width. In the end, you should have determined what amount of sugar reliably gives you "amber", and what amount produces different results, which you sometimes classify as "amber" and sometimes as "boiled". 

A word on why this complicated approach is needed

You said that you dislike following recipes. But at the same time, you want to get to a well defined end state, and do so under a constraint (minimal amount of sugar). 

You have a few choices how you can approach cooking. If you want freedom and creativity, and are happy with a random outcome, you can do random experiments. A little bit of kitchen experience will be enough to get good results most of the time. Alternatively, if you want a very specific outcome, you can strictily follow a recipe where somebody did all the work needed to find out how to achieve it. If you have decided that you will not follow a recipe, and if you have set your mind on a specific outcome, you have no choice but to invest the work needed to develop the recipe for that outcome. And the more constraints you have, the more work it is. 

If you hoped that it is possible to take a look at the outcome of an experiment and predict the conditions which will give you the same results under a constraint - no, the chemistry and physics of cooking is way too complicated for doing that. The experimental approach is long and boring, but it is still the most efficient (and actually the most effective) one you can take.