Doubt to use accuracy or macro f1 measure in an unbalanced classification task I have a multi-class classification task where the organizers said that the final results will be using the Accuracy measure. 

The provided data is unbalanced, and I don't have an idea about the test set (is it balanced or not), but I think it will be balanced since they use accuracy.

Anyway ..

My question: Is it a good idea to tune my system using F1-macro rather than Accuracy? since the training data is unbalanced.

or it's better to use the Accuracy?
using accuracy for unbalance data means that correct classification for the most populous class members is more important than others. If the importance of correct classifying for all data records is equal in your problem accuracy is one of the worst choices.

There are some other good choices beside F1-macro which can be more helpful. Some of these metrics are as follows:

"Kappa", "SOA1(Landis &amp; Koch)", "SOA2(Fleiss)", "SOA3(Altman)", "SOA4(Cicchetti)", "CEN",
"MCEN", "MCC", "J", "Overall J", "Overall MCC", "Overall CEN", "Overall MCEN", "AUC",
"AUCI", "G", "DP", "DPI", "GI"


Disclaimer: 

If you use python, PyCM module can help you to find out these metrics.

Here is a simple code to get the recommended parameters from this module:

&gt;&gt;&gt; from pycm import *

&gt;&gt;&gt; cm = ConfusionMatrix(matrix={"Class1": {"Class1": 1, "Class2":2}, "Class2": {"Class1": 0, "Class2": 5}})  

&gt;&gt;&gt; print(cm.recommended_list)
["Kappa", "SOA1(Landis &amp; Koch)", "SOA2(Fleiss)", "SOA3(Altman)", "SOA4(Cicchetti)", "CEN", "MCEN", "MCC", "J", "Overall J", "Overall MCC", "Overall CEN", "Overall MCEN", "AUC", "AUCI", "G", "DP", "DPI", "GI"]


After that, each of these parameters you want to use as the loss function can be used as follows:

&gt;&gt;&gt; y_pred = model.predict      #the prediction of the implemented model

&gt;&gt;&gt; y_actu = data.target        #data labels

&gt;&gt;&gt; cm = ConfusionMatrix(y_actu, y_pred)

&gt;&gt;&gt; loss = cm.Kappa             #or any other parameter (Example: cm.SOA1)