问:
How to implement a Flink Event Time Trigger that emits after no events recieved for X minutes
I'm struggling a bit understanding how Flink Triggers work. My datastream contains events with a sessionId that I aggregated based on that sessionId. Each session will contain a Started and a Ended event however some times the Ended event will be lost.

In order to handle this I've set up a Trigger that will emit the aggregated session whenever the ended event is processed. But in the case that no events arrive from that session for 2 minutes I want to emit whatever we have aggregated so far (our apps that send the events send heartbeats every minute so if we don't get any events the session is considered lost).

I've set up the following trigger function:

```<code>public class EventTimeProcessingTimeTrigger extends Trigger<HashMap, TimeWindow> {
    private final long sessionTimeout;
    private long lastSetTimer;

    // Max session length set to 1 day
    public static final long MAX_SESSION_LENGTH = 1000l * 86400l;

    // End session events
    private static ImmutableSet<String> endSession = ImmutableSet.<String>builder()
            .add("Playback.Aborted")
            .add("Playback.Completed")
            .add("Playback.Error")
            .add("Playback.StartAirplay")
            .add("Playback.StartCasting")
            .build();

    public EventTimeProcessingTimeTrigger(long sessionTimeout) {
        this.sessionTimeout = sessionTimeout;
    }

    @Override
    public TriggerResult onElement(HashMap element, long timestamp, TimeWindow window, TriggerContext ctx) throws Exception {
        lastSetTimer = ctx.getCurrentProcessingTime() + sessionTimeout;
        ctx.registerProcessingTimeTimer(lastSetTimer);

        if(endSession.contains(element.get(Field.EVENT_TYPE))) {
            return TriggerResult.FIRE_AND_PURGE;
        }

        return TriggerResult.CONTINUE;
    }

    @Override
    public TriggerResult onProcessingTime(long time, TimeWindow window, TriggerContext ctx) throws Exception {
        return TriggerResult.FIRE_AND_PURGE;
    }

    @Override
    public TriggerResult onEventTime(long time, TimeWindow window, TriggerContext ctx) throws Exception {
        return time == window.maxTimestamp() ?
                TriggerResult.FIRE_AND_PURGE :
                TriggerResult.CONTINUE;
    }

    @Override
    public void clear(TimeWindow window, TriggerContext ctx) throws Exception {
        ctx.deleteProcessingTimeTimer(lastSetTimer);
    }

    @Override
    public boolean canMerge() {
        return true;
    }

    @Override
    public void onMerge(TimeWindow window,
                        OnMergeContext ctx) {
        ctx.registerProcessingTimeTimer(ctx.getCurrentProcessingTime() + sessionTimeout);
    }
}
</code>```

In order to set watermarks for the events I use the watermarks set by the apps since appEventTime might not be the same as wallClock on the server. I extract watermarks like this:

```<code>DataStream<HashMap> playerEvents = env
                .addSource(kafkaConsumerEvents, "playerEvents(Kafka)")
                .name("Read player events from Kafka")
                .uid("Read player events from Kafka")
                .map(json -> DECODER.decode(json, TypeToken.of(HashMap.class))).returns(HashMap.class)
                .name("Map Json to HashMap")
                .uid("Map Json to HashMap")
                .assignTimestampsAndWatermarks(new BoundedOutOfOrdernessTimestampExtractor<HashMap>(org.apache.flink.streaming.api.windowing.time.Time.seconds(30))
                {
                    @Override
                    public long extractTimestamp(HashMap element)
                    {
                        long timestamp = 0L;
                        Object timestampAsObject = (Object) element.get("CanonicalTime");
                        timestamp = (long)timestampAsObject;
                        return timestamp;
                    }
                })
                .name("Add CanonicalTime as timestamp")
                .uid("Add CanonicalTime as timestamp");
</code>```

Now what I find strange is that when I run the code in debug and set a breakpoint in the clear function of the Trigger it constantly gets called. Even when no FIRE_AND_PURGE point is reached in the Trigger. So it feels like I've completely misunderstood how the Trigger is supposed to work. And that my implementation is not at all doing what I think it's doing.

I guess my question is, when should clear be called by the Trigger? And is this the correct way to implement a combined EventTimeTrigger and ProcessingTimeTrigger?

Thankful for all the help I can get.

UPDATE 1: (2020-05-29)

In order to provide some more information about how things are setup.
I set up my environment as follows:

```<code>final StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
        env.setRestartStrategy(RestartStrategies.failureRateRestart(60, Time.of(60, TimeUnit.MINUTES), Time.of(60, TimeUnit.SECONDS)));
        env.enableCheckpointing(5000);
        env.getCheckpointConfig().setMinPauseBetweenCheckpoints(2000);
env.getCheckpointConfig().setCheckpointingMode(CheckpointingMode.EXACTLY_ONCE);
        env.setStreamTimeCharacteristic(TimeCharacteristic.EventTime);
</code>```

So I use EventTime for the entire stream.
I then create the windows like this:

```<code>DataStream<PlayerSession> playerSessions = sideEvents
                .keyBy((KeySelector<HashMap, String>) event -> (String) event.get(Field.SESSION_ID))
                .window(ProcessingTimeSessionWindows.withGap(org.apache.flink.streaming.api.windowing.time.Time.minutes(5)))
                .trigger(new EventTimeProcessingTimeTrigger(SESSION_TIMEOUT))
                .aggregate(new SessionAggregator())
                .name("Aggregate events into sessions")
                .uid("Aggregate events into sessions");
</code>```

答:
This situation is complex. I hesitate to predict exactly what this code will do, but I can explain some of what’s going on.

Point 1: you have set the time characteristic to event time, arranged for timestamps and watermarks, and implemented an onEventTime callback in your Trigger. But nowhere are you creating an event time timer. Unless I've missed something, nothing is actually using event time or watermarks. You haven't implemented an event time trigger, and I would not expect that onEventTime will ever be called.

Point 2: Your trigger doesn't need to call clear. Flink takes care of calling clear on triggers as part of purging windows.

Point 3: Your trigger is trying to fire and purge the window repeatedly, which doesn't seem right. I say this because you are creating a new processing time timer for every element, and when each timer fires, you are firing and purging the window. You can fire the window as often as you like, but you can only purge the window once, after which it is gone.

Point 4: Session windows are a special kind of window, known as merging windows. When sessions merge (which happens all the time, as events arrive), their triggers are merged, and one of them gets cleared. This is why you see clear being called so frequently.

Suggestion: since you have once-a-minute keepalives, and intend to close sessions after 2 minutes of inactivity, it seems like you could set the session gap to be 2 minutes, and that would avoid a fair bit of what's making things so complex. Let the session windows do what they're designed to do.

Assuming that will work, then you could simple extend Flink's ProcessingTimeTrigger and override its onElement method to do this:

@Override
public TriggerResult onElement(HashMap element, long timestamp, TimeWindow window, TriggerContext ctx) throws Exception {

    if (endSession.contains(element.get(Field.EVENT_TYPE))) {
        return TriggerResult.FIRE_AND_PURGE;
    }

    return super(element, timestamp, window, ctx);
}
```

In this fashion the window will be triggered after two minutes of inactivity, or by an explicit session-ending event.

You should be able to simply inherit the rest of ProcessingTimeTrigger's behavior.

If you want to use event time, then use EventTimeTrigger as the superclass, and you'll have to find a way to make sure that your watermarks make progress even when the stream becomes idle. See this answer for how to handle that.

Body:
I'm struggling a bit understanding how Flink Triggers work. My datastream contains events with a sessionId that I aggregated based on that sessionId. Each session will contain a Started and a Ended event however some times the Ended event will be lost.

In order to handle this I've set up a Trigger that will emit the aggregated session whenever the ended event is processed. But in the case that no events arrive from that session for 2 minutes I want to emit whatever we have aggregated so far (our apps that send the events send heartbeats every minute so if we don't get any events the session is considered lost).

I've set up the following trigger function:

public class EventTimeProcessingTimeTrigger extends Trigger<HashMap, TimeWindow> {
    private final long sessionTimeout;
    private long lastSetTimer;

    // Max session length set to 1 day
    public static final long MAX_SESSION_LENGTH = 1000l * 86400l;

    // End session events
    private static ImmutableSet<String> endSession = ImmutableSet.<String>builder()
            .add("Playback.Aborted")
            .add("Playback.Completed")
            .add("Playback.Error")
            .add("Playback.StartAirplay")
            .add("Playback.StartCasting")
            .build();

    public EventTimeProcessingTimeTrigger(long sessionTimeout) {
        this.sessionTimeout = sessionTimeout;
    }

    @Override
    public TriggerResult onElement(HashMap element, long timestamp, TimeWindow window, TriggerContext ctx) throws Exception {
        lastSetTimer = ctx.getCurrentProcessingTime() + sessionTimeout;
        ctx.registerProcessingTimeTimer(lastSetTimer);

        if(endSession.contains(element.get(Field.EVENT_TYPE))) {
            return TriggerResult.FIRE_AND_PURGE;
        }

        return TriggerResult.CONTINUE;
    }

    @Override
    public TriggerResult onProcessingTime(long time, TimeWindow window, TriggerContext ctx) throws Exception {
        return TriggerResult.FIRE_AND_PURGE;
    }

    @Override
    public TriggerResult onEventTime(long time, TimeWindow window, TriggerContext ctx) throws Exception {
        return time == window.maxTimestamp() ?
                TriggerResult.FIRE_AND_PURGE :
                TriggerResult.CONTINUE;
    }

    @Override
    public void clear(TimeWindow window, TriggerContext ctx) throws Exception {
        ctx.deleteProcessingTimeTimer(lastSetTimer);
    }

    @Override
    public boolean canMerge() {
        return true;
    }

    @Override
    public void onMerge(TimeWindow window,
                        OnMergeContext ctx) {
        ctx.registerProcessingTimeTimer(ctx.getCurrentProcessingTime() + sessionTimeout);
    }
}

In order to set watermarks for the events I use the watermarks set by the apps since appEventTime might not be the same as wallClock on the server. I extract watermarks like this:

DataStream<HashMap> playerEvents = env
                .addSource(kafkaConsumerEvents, "playerEvents(Kafka)")
                .name("Read player events from Kafka")
                .uid("Read player events from Kafka")
                .map(json -> DECODER.decode(json, TypeToken.of(HashMap.class))).returns(HashMap.class)
                .name("Map Json to HashMap")
                .uid("Map Json to HashMap")
                .assignTimestampsAndWatermarks(new BoundedOutOfOrdernessTimestampExtractor<HashMap>(org.apache.flink.streaming.api.windowing.time.Time.seconds(30))
                {
                    @Override
                    public long extractTimestamp(HashMap element)
                    {
                        long timestamp = 0L;
                        Object timestampAsObject = (Object) element.get("CanonicalTime");
                        timestamp = (long)timestampAsObject;
                        return timestamp;
                    }
                })
                .name("Add CanonicalTime as timestamp")
                .uid("Add CanonicalTime as timestamp");

Now what I find strange is that when I run the code in debug and set a breakpoint in the clear function of the Trigger it constantly gets called. Even when no FIRE_AND_PURGE point is reached in the Trigger. So it feels like I've completely misunderstood how the Trigger is supposed to work. And that my implementation is not at all doing what I think it's doing.

I guess my question is, when should clear be called by the Trigger? And is this the correct way to implement a combined EventTimeTrigger and ProcessingTimeTrigger?

Thankful for all the help I can get.

UPDATE 1: (2020-05-29)

In order to provide some more information about how things are setup.
I set up my environment as follows:

final StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
        env.setRestartStrategy(RestartStrategies.failureRateRestart(60, Time.of(60, TimeUnit.MINUTES), Time.of(60, TimeUnit.SECONDS)));
        env.enableCheckpointing(5000);
        env.getCheckpointConfig().setMinPauseBetweenCheckpoints(2000);
env.getCheckpointConfig().setCheckpointingMode(CheckpointingMode.EXACTLY_ONCE);
        env.setStreamTimeCharacteristic(TimeCharacteristic.EventTime);

So I use EventTime for the entire stream.
I then create the windows like this:

DataStream<PlayerSession> playerSessions = sideEvents
                .keyBy((KeySelector<HashMap, String>) event -> (String) event.get(Field.SESSION_ID))
                .window(ProcessingTimeSessionWindows.withGap(org.apache.flink.streaming.api.windowing.time.Time.minutes(5)))
                .trigger(new EventTimeProcessingTimeTrigger(SESSION_TIMEOUT))
                .aggregate(new SessionAggregator())
                .name("Aggregate events into sessions")
                .uid("Aggregate events into sessions");

Body:
This situation is complex. I hesitate to predict exactly what this code will do, but I can explain some of what’s going on.

Point 1: you have set the time characteristic to event time, arranged for timestamps and watermarks, and implemented an onEventTime callback in your Trigger. But nowhere are you creating an event time timer. Unless I've missed something, nothing is actually using event time or watermarks. You haven't implemented an event time trigger, and I would not expect that onEventTime will ever be called.

Point 2: Your trigger doesn't need to call clear. Flink takes care of calling clear on triggers as part of purging windows.

Point 3: Your trigger is trying to fire and purge the window repeatedly, which doesn't seem right. I say this because you are creating a new processing time timer for every element, and when each timer fires, you are firing and purging the window. You can fire the window as often as you like, but you can only purge the window once, after which it is gone.

Point 4: Session windows are a special kind of window, known as merging windows. When sessions merge (which happens all the time, as events arrive), their triggers are merged, and one of them gets cleared. This is why you see clear being called so frequently.

Suggestion: since you have once-a-minute keepalives, and intend to close sessions after 2 minutes of inactivity, it seems like you could set the session gap to be 2 minutes, and that would avoid a fair bit of what's making things so complex. Let the session windows do what they're designed to do.

Assuming that will work, then you could simple extend Flink's ProcessingTimeTrigger and override its onElement method to do this:

@Override
public TriggerResult onElement(HashMap element, long timestamp, TimeWindow window, TriggerContext ctx) throws Exception {

    if (endSession.contains(element.get(Field.EVENT_TYPE))) {
        return TriggerResult.FIRE_AND_PURGE;
    }

    return super(element, timestamp, window, ctx);
}

In this fashion the window will be triggered after two minutes of inactivity, or by an explicit session-ending event.

You should be able to simply inherit the rest of ProcessingTimeTrigger's behavior.

If you want to use event time, then use EventTimeTrigger as the superclass, and you'll have to find a way to make sure that your watermarks make progress even when the stream becomes idle. See this answer for how to handle that.


来源: stackexchange

问:
The compiler is complaining about my default parameters?
I'm having trouble with this piece of code , after i took this class from the main.cpp file and splitted it in to .h and .cpp the compiler started complaining about the default parameters i was using in a void.

/* PBASE.H */
    class pBase : public sf::Thread {
private:
    bool Running;

public:
    sf::Mutex Mutex;
    WORD OriginalColor;
    pBase(){
        Launch();
        Running = true;
        OriginalColor = 0x7;
    }
    void progressBar(int , int);
    bool key_pressed();
    void setColor( int );
    void setTitle( LPCWSTR );
    bool test_connection(){
        if(Running == false){
            return 0;
        }
        else{
            return 1;
        }
    return 0;
    }
    void Stop(){
        Running = false;
        if(Running == false) Wait();
    }
};
```

<hr>

<pre class="lang-c prettyprint-override"><code>    /* PBASE.CPP */

    // ... other stuff above

    void pBase::setColor( int _color = -1){
        if(_color == -1){
             SetConsoleTextAttribute( GetStdHandle( STD_OUTPUT_HANDLE ),FOREGROUND_INTENSITY | OriginalColor);
             return;
        }
        SetConsoleTextAttribute( GetStdHandle( STD_OUTPUT_HANDLE ),FOREGROUND_INTENSITY | _color);

}
</code>```

And the error , taken from VC2010

Error 4   error C2572: 'pBase::setColor' : redefinition of default parameter : parameter 1

答:
You have to specify the default values for the arguments only in the declaration but not in the definition.

```<code> class pBase : public sf::Thread {
     // ....
     void setColor( int _color = -1 );
     // ....
 } ;

 void pBase:: setColor( int _color )
 {
     // ....
 }
</code>```

The default value for an member function's argument can either go in declaration or definition but not both. Quote from ISO/IEC 14882:2003(E) 8.3.6

6)       Except for member functions of class templates, the default arguments in a member function definition that appears outside of the class definition are added to the set of default arguments provided by the member function declaration in the class definition. Default arguments for a member function of a class template shall be specified on the initial declaration of the member function within the class template. [Example:

```<code>class C {
    void f(int i = 3);
    void g(int i, int j = 99);
};

void C::f(int i = 3)   // error: default argument already
{ }                    // specified in class scope

void C::g(int i = 88, int j)    // in this translation unit,
{ }                             // C::g can be called with no argument
</code>```

—end example]

According to the standard provided example, it should actually work the way you did. Unless you have done like this, you shouldn't actually get the error. I amn't sure why it actually worked in your case with my solution. Probably something visual studio related, I guess.

ContentLicense: CC BY-SA 3.0

Body:
I'm having trouble with this piece of code , after i took this class from the main.cpp file and splitted it in to .h and .cpp the compiler started complaining about the default parameters i was using in a void.

/* PBASE.H */
    class pBase : public sf::Thread {
private:
    bool Running;

public:
    sf::Mutex Mutex;
    WORD OriginalColor;
    pBase(){
        Launch();
        Running = true;
        OriginalColor = 0x7;
    }
    void progressBar(int , int);
    bool key_pressed();
    void setColor( int );
    void setTitle( LPCWSTR );
    bool test_connection(){
        if(Running == false){
            return 0;
        }
        else{
            return 1;
        }
    return 0;
    }
    void Stop(){
        Running = false;
        if(Running == false) Wait();
    }
};

    /* PBASE.CPP */

    // ... other stuff above

    void pBase::setColor( int _color = -1){
        if(_color == -1){
             SetConsoleTextAttribute( GetStdHandle( STD_OUTPUT_HANDLE ),FOREGROUND_INTENSITY | OriginalColor);
             return;
        }
        SetConsoleTextAttribute( GetStdHandle( STD_OUTPUT_HANDLE ),FOREGROUND_INTENSITY | _color);

}

And the error , taken from VC2010

Error 4   error C2572: 'pBase::setColor' : redefinition of default parameter : parameter 1

Body:
You have to specify the default values for the arguments only in the declaration but not in the definition.

 class pBase : public sf::Thread {
     // ....
     void setColor( int _color = -1 );
     // ....
 } ;

 void pBase:: setColor( int _color )
 {
     // ....
 }

The default value for an member function's argument can either go in declaration or definition but not both. Quote from ISO/IEC 14882:2003(E) 8.3.6

6)       Except for member functions of class templates, the default arguments in a member function definition that appears outside of the class definition are added to the set of default arguments provided by the member function declaration in the class definition. Default arguments for a member function of a class template shall be specified on the initial declaration of the member function within the class template. [Example:

class C {
    void f(int i = 3);
    void g(int i, int j = 99);
};

void C::f(int i = 3)   // error: default argument already
{ }                    // specified in class scope

void C::g(int i = 88, int j)    // in this translation unit,
{ }                             // C::g can be called with no argument

—end example]

According to the standard provided example, it should actually work the way you did. Unless you have done like this, you shouldn't actually get the error. I amn't sure why it actually worked in your case with my solution. Probably something visual studio related, I guess.


问:
MySQL making a poor optimization choice with ORDER by query and the EXPLAIN is incorrect
I am having a problem where mysql 5.6 is not using an index and causing very slow queries.

Table named people with the following fields: id, group_id, city_id, ...

Index that should be usable for group_id: (group_id, city_id)

The query is:

```<code>SELECT  people.* FROM people WHERE people.group_id = 12345 ORDER BY people.id ASC LIMIT 10 OFFSET 0;
</code>```

The EXPLAIN for this query shows it not using the index and scanning 9324 rows:

```<code>possible_keys: index_people_on_group_id_and_city_id
key: PRIMARY
rows: 9324
extra: Using where
</code>```

However, my slow query log shows the rows examined is huge:

```<code>Query_time: 16.254633  Lock_time: 0.000038 Rows_sent: 10  Rows_examined: 7429457
</code>```

Why does it not use the index?

Why is the optimizer so wrong? Do I need to somehow rebuild the metadata so that it realizes it has to scan more than 9324 rows and can then make a better choice?

I don't know where the number 9324 comes from. There is more than 10k records for group_id 12345.

edit: I discovered the size of the LIMIT is important here. With a LIMIT of 100 it uses the keys. With a limit of 10 it thinks incorrectly that it has to scan 9324 rows. With a limit of 1 it thinks incorrectly that it has to scan 932 rows.

答:
I have discovered this is likely the result of a bug fixed in MySQL 5.7.6.

Release Notes
(https://dev.mysql.com/doc/relnotes/mysql/5.7/en/news-5-7-6.html)

Bug: https://bugs.mysql.com/bug.php?id=73837

For queries that combine ORDER BY with LIMIT, the optimizer may switch
  to an index that applies to the ORDER BY. In some cases, the decision
  to switch was based on a heuristic rather than on cost. The optimizer
  now uniformly makes the decision whether to switch on a cost basis.
  This should result in better performanance when switching would cause
  a query to read an entire index or a large part of it to find
  qualifying rows.

ContentLicense: CC BY-SA 4.0

Body:
I am having a problem where mysql 5.6 is not using an index and causing very slow queries.

Table named people with the following fields: id, group_id, city_id, ...

Index that should be usable for group_id: (group_id, city_id)

The query is:

SELECT  people.* FROM people WHERE people.group_id = 12345 ORDER BY people.id ASC LIMIT 10 OFFSET 0;

The EXPLAIN for this query shows it not using the index and scanning 9324 rows:

possible_keys: index_people_on_group_id_and_city_id
key: PRIMARY
rows: 9324
extra: Using where

However, my slow query log shows the rows examined is huge:

Query_time: 16.254633  Lock_time: 0.000038 Rows_sent: 10  Rows_examined: 7429457

Why does it not use the index?

Why is the optimizer so wrong? Do I need to somehow rebuild the metadata so that it realizes it has to scan more than 9324 rows and can then make a better choice?

I don't know where the number 9324 comes from. There is more than 10k records for group_id 12345.

edit: I discovered the size of the LIMIT is important here. With a LIMIT of 100 it uses the keys. With a limit of 10 it thinks incorrectly that it has to scan 9324 rows. With a limit of 1 it thinks incorrectly that it has to scan 932 rows.


问:
Setting useState within another useState
I am attempting to set the time (mm:ss) by taking separate inputs for mins and secs with one use of useState and combining them with another for keeping track of the overall form data, however I cannot seem to get the combined string to push into the form data...

I think I may be setting the data incorrectly somehow...

```<code>const Modal = ({show, handleClose, handleAdd, children}) => {
    const today = new Date().toISOString().substr(0, 10);
    let initialFormState = {
        date: today,
        distance: '5.23',
        time: ''
    };
    let initialTimeState = {
        mins: 0,
        secs: 0
    };

    const [formData, setFormData] = useState(initialFormState);
    const [timeData, setTimeData] = useState(initialTimeState);

    const submitData = event => {
        event.preventDefault();
        console.log({timeData});

        // Add correctly formatted time to main formData
        const {mins, secs} = timeData;
        const time = `${mins}:${secs}`;
        setFormData({...formData, 'time': time});

        console.log(formData); // Here I can see that the 'time' field is empty...

    }

    return (
       ...
    );
};

export default Modal;
</code>```

答:
setFormData changes are happening asynchronously, unfortunately you won't see the modification immediately. Instead you can use useEffect hook to catch changes of your state.

Try as the following instead:

```<code>useEffect(() => {
   console.log(formData);
}, [formData]);
</code>```

Read further about useEffect hook at Using the Effect Hook link.

Body:
I am attempting to set the time (mm:ss) by taking separate inputs for mins and secs with one use of useState and combining them with another for keeping track of the overall form data, however I cannot seem to get the combined string to push into the form data...

I think I may be setting the data incorrectly somehow...

const Modal = ({show, handleClose, handleAdd, children}) => {
    const today = new Date().toISOString().substr(0, 10);
    let initialFormState = {
        date: today,
        distance: '5.23',
        time: ''
    };
    let initialTimeState = {
        mins: 0,
        secs: 0
    };

    const [formData, setFormData] = useState(initialFormState);
    const [timeData, setTimeData] = useState(initialTimeState);

    const submitData = event => {
        event.preventDefault();
        console.log({timeData});

        // Add correctly formatted time to main formData
        const {mins, secs} = timeData;
        const time = `${mins}:${secs}`;
        setFormData({...formData, 'time': time});

        console.log(formData); // Here I can see that the 'time' field is empty...

    }

    return (
       ...
    );
};

export default Modal;

Body:
setFormData changes are happening asynchronously, unfortunately you won't see the modification immediately. Instead you can use useEffect hook to catch changes of your state.

Try as the following instead:

useEffect(() => {
   console.log(formData);
}, [formData]);

Read further about useEffect hook at Using the Effect Hook link.


问:
The method '[]' was called on null. Receiver: null
To isolate the issue,  I created a new code, a new program, and a new machine.
Same results, it just doesn't want to print any value that has  is after  "quotes and has "anythingPrice"

Again I can give you a temp key and you can fully run the code. on your side.

Please let me know.

```<code>import 'dart:async';
import 'dart:convert';
import 'package:flutter/foundation.dart';
import 'package:flutter/material.dart';
import 'package:http/http.dart' as http;
//import 'package:flutter_web/material.dart';
//import 'package:flutter/main.dart' as app;
//import 'package:flutter_web_ui/ui.dart' as ui;

Future<List<Photo>> fetchPhotos(http.Client client) async {
  final response =
 // await client.get('https://cloud.iexapis.com/stable/stock/market/batch?symbols=CNPF,ICOL,PGX,MORL,EMLC,GYLD,IPFF,SMB,YDIV,YYY,PEY,IEI,TLH,DIA,BSJF,MONY&types=quote,stats&token=');

  await client.get('https://cloud.iexapis.com/stable/stock/market/batch?symbols=PGX,MORL,EMLC,GYLD,IPFF,SMB,YDIV,YYY,PEY,PGF,SNLN,DVHL,AUNZ,MDIV,PHB,PFXF,FPE,TIPX,MLN,SDIV,SIPE,SEMF,EMAG,IEMF,EU,BSCM,LEMF,BSCL,BSCK,BSCJ,BSCI,DSUM,BSCF,KBWD,PCEF,BSCG,BSCH,HYEM,ITM,BKLN,PGHY,IBCE,PZT,IJNK,XMPT,THHY,QYLD,IHY,IYLD,SHYD,PRB,DVHI,GOVT,BSJJ,BSJK,RIGS,INY,IBCD,PZA,PFIG,IBCC,BSCE,PVI,MBG,CXA,FWDB,DIV,CHLC,BSJI,FLTR,PWZ,MUAH,IBDD,PICB,IBDB,BSJH,RVNU,BSJG,IBDC,XOVR,BSJE,EBND,PCY,BWX,SJNK,DES,ANGL,BAB,SST,EMCD,CBND,HYD,PLW,SCPB,FLRN,BWZ,KBWY,ELD,IBND,AOK,RWXL,ITR,DON,DGRS,JNK,TZD,EPU,PFF,HYLD,GGOV,ULST,SPHD,HFIN,DGRW,LWC,ALD,LEMB,AGND,ITIP,EMHY,QLTC,TFI,SRLN,HYLS,SHM,GNMA,CEMB,IGOV,MINC,AGZD,FTSL,SPLV,GHYG,ISTB,ENGN,CMBS,GTIP,SCHO,SMMU,QLTA,SCHZ,HSPX,TUZ,GIY,GSY,TFLO,VMBS,CWB,FLOT,STPZ,MUAC,SCHR,BABZ,HYXU,GMTB,MUNI,WIP,MUAD,QLTB,SCHP,GMMB,MUAE,IPE,NYF,BNDX,MUAF,HYMB,TIPZ,CMF,LAG,ILTB,ITE,CLY,VGSH,VGIT,LTPZ,BABS,HYHG,EMCB,DVYL,VGLT,DHS,TLO,VWOB,IGHG,GLCB,RAVI,VCSH,BSV,BND,BIV,ISHG,SDYL,VCIT,SHY,BLV,HYG,VCLT,TENZ,DTN,BIL,AUD,DLN,DTD,FTSD,CAD,STIP,CORP,HYS,HOLD,TRSY,IEF,IBDA,COBO,BOND,MINT,MBB,CSJ,AGG,SUB,EMB,CIU,GVI,MUB,GBF,TIP,AGZ,CFT,LQD,TLT,IEI,TLH,DIA,BSJF,MONY&types=stats,quote&token=');
  //https://cloud.iexapis.com/v1/stock/market/batch?&types=price&symbols=aapl,fb,tsla&token=YOUR_TOKEN_HERE
  //https://cloud.iexapis.com/v1/stock/market/batch?&types=quote&symbols=aapl,fb,tsla&token=
  // Use the compute function to run parsePhotos in a separate isolate.
  //https://cloud.iexapis.com/stable/stock/market/batch?symbols=CNPF,ICOL,PGX,MORL,EMLC,GYLD,IPFF,SMB,YDIV,YYY,PEY,PGF,SNLN,DVHL,AUNZ,MDIV,PHB,PFXF,FPE,TIPX,MLN,SDIV,SIPE,SEMF,EMAG,IEMF,EU,BSCM,LEMF,BSCL,BSCK,BSCJ,BSCI,DSUM,BSCF,KBWD,PCEF,BSCG,BSCH,HYEM,ITM,BKLN,PGHY,IBCE,PZT,IJNK,XMPT,THHY,QYLD,IHY,IYLD,SHYD,PRB,DVHI,GOVT,BSJJ,BSJK,RIGS,INY,IBCD,PZA,PFIG,IBCC,BSCE,PVI,MBG,CXA,FWDB,DIV,CHLC,BSJI,FLTR,PWZ,MUAH,IBDD,PICB,IBDB,BSJH,RVNU,BSJG,IBDC,XOVR,BSJE,EBND,PCY,BWX,SJNK,DES,ANGL,BAB,SST,EMCD,CBND,HYD,PLW,SCPB,FLRN,BWZ,KBWY,ELD,IBND,AOK,RWXL,ITR,DON,DGRS,JNK,TZD,EPU,PFF,HYLD,GGOV,ULST,SPHD,HFIN,DGRW,LWC,ALD,LEMB,AGND,ITIP,EMHY,QLTC,TFI,SRLN,HYLS,SHM,GNMA,CEMB,IGOV,MINC,AGZD,FTSL,SPLV,GHYG,ISTB,ENGN,CMBS,GTIP,SCHO,SMMU,QLTA,SCHZ,HSPX,TUZ,GIY,GSY,TFLO,VMBS,CWB,FLOT,STPZ,MUAC,SCHR,BABZ,HYXU,GMTB,MUNI,WIP,MUAD,QLTB,SCHP,GMMB,MUAE,IPE,NYF,BNDX,MUAF,HYMB,TIPZ,CMF,LAG,ILTB,ITE,CLY,VGSH,VGIT,LTPZ,BABS,HYHG,EMCB,DVYL,VGLT,DHS,TLO,VWOB,IGHG,GLCB,RAVI,VCSH,BSV,BND,BIV,ISHG,SDYL,VCIT,SHY,BLV,HYG,VCLT,TENZ,DTN,BIL,AUD,DLN,DTD,FTSD,CAD,STIP,CORP,HYS,HOLD,TRSY,IEF,IBDA,COBO,BOND,MINT,MBB,CSJ,AGG,SUB,EMB,CIU,GVI,MUB,GBF,TIP,AGZ,CFT,LQD,TLT,IEI,TLH,DIA,BSJF,MONY&types=quote,stats&token=
  return compute(parsePhotos, response.body);
}

// A function that converts a response body into a List<Photo>.
List<Photo> parsePhotos(String responseBody) {
  //final parsed = json.decode(responseBody).cast<Map<String,dynamic>>();

  //return parsed.map<Photo>((json) => Photo.fromJson(json)).toList();

  dynamic Obj = json.decode(responseBody);
  debugPrint(responseBody, wrapWidth: 8192);
  print(Obj.length);

  List<Photo> photoList = [];
  Obj.forEach((k, v) => photoList.add(Photo(k,v)));

  return photoList;
}

class Photo {
  String symbol;
  //String companyName;
  dynamic data;
 // dynamic iexClose;
 // dynamic quote;
  Photo(this.symbol ,this.data);

}

void main() => runApp(MyApp());

class MyApp extends StatelessWidget {
  @override

  Widget build(BuildContext context) {

    final appTitle = 'Monthly Paying ETFs';

    return MaterialApp(
      title: appTitle,
      home: MyHomePage(title: appTitle),

    );
  }
}

class MyHomePage extends StatelessWidget {

  final String title;

  MyHomePage({Key key, this.title}) : super(key: key);

  @override
  Widget build(BuildContext context) {

    return Scaffold(

        drawer: Drawer(
          // Add a ListView to the drawer. This ensures the user can scroll
          // through the options in the drawer if there isn't enough vertical
          // space to fit everything.
          child: ListView(
            // Important: Remove any padding from the ListView.
            padding: EdgeInsets.zero,
            children: <Widget>[
              DrawerHeader(
                child: Text('About'),
                decoration: BoxDecoration(
                  color: Colors.grey,
                ),
              ),
              ListTile(
                title: Text('This app is to provide the list of ETF s that pay monthly dividend'),

                onTap: () {
                  // Update the state of the app
                  // ...
                  // Then close the drawer
                  Navigator.pop(context);
                },

              ),
              ListTile(
                title: Text('for questions or feedback please contact me at  '),

                onTap: () {
                  // Update the state of the app
                  // ...
                  // Then close the drawer
                  Navigator.pop(context);
                },

              ),

            ],
          ),
        ),

        appBar: AppBar(
        title: Text(title),
          backgroundColor: Colors.greenAccent,
      ),

      body: FutureBuilder<List<Photo>>(

        future: fetchPhotos(http.Client()),

        builder: (context, snapshot) {
          if (snapshot.hasError) print(snapshot.error);

          return snapshot.hasData
              ? PhotosList(photos: snapshot.data)
              : Center(child: CircularProgressIndicator());

        },
      ),
    );
  }
}

class PhotosList extends StatefulWidget {
  final List photos;

  PhotosList({Key key, this.photos})
      : assert(photos != null),
        super(key: key);

  @override
  _PhotosListState createState() => _PhotosListState();
}

class _PhotosListState extends State<PhotosList> {
  @override

  Widget build(BuildContext context) {

    return bodyData();

  }

  SingleChildScrollView bodyData() =>

    SingleChildScrollView(
  scrollDirection: Axis.vertical,

      // padding: EdgeInsets.all(0.0),
       //padding: const EdgeInsets.symmetric(
       //  padding: new EdgeInsets.fromLTRB(0.001, .001, 1.0, 1.0),
       padding: EdgeInsets.all(0.05),

 // child: FittedBox(fit:BoxFit.contain,

  child:SingleChildScrollView(
    scrollDirection: Axis.horizontal,

  child:
  FittedBox(
  fit: BoxFit.fitHeight,
  child: DataTable(
         // sortColumnIndex: 1,
        //  sortAscending: true,

          columns: <DataColumn>[
            DataColumn(
              label: Text("Company"),
              onSort: (_, __) {
                setState(() {
                  widget.photos.sort((a, b) =>
                      a.data["quote"]["companyName"]
                          .compareTo(b.data["quote"]["companyName"]));
                });
              },
            ),

            DataColumn(
              label: Text("ttmDivRate"),
              numeric: true,
              onSort:   (_,__) {
                setState(() {
                  widget.photos.sort((a, b) =>
                      a.data["quote"]["ttmDividendRate"]
                          .compareTo(b.data["quote"]["ttmDividendRate"]));
                });
              },
            ),

            DataColumn(
              label: Text("Price"),
              numeric: true,
              onSort: (_, __) {
                setState(() {
                  widget.photos.sort((a, b) => a.data ["quote"]["latestPrice"]
                      .compareTo(b.data["quote"]["latestPrice"]));
                });
              },
            ),

          ],

          rows: widget.photos

              .map(
                (photo) =>

                DataRow(
                  cells: [

                    DataCell(
                      Text('${photo.data["stats"]["companyName"] ?? ""}'),
                    ),

                    DataCell(
                      Text('${photo.data["stats"]["ttmDividendRate"] ?? ""}'),
                   ),
//["quote"]["stats"]
                    DataCell(
                      Text('${photo.data["quote"]["stats"]["latestPrice"] ?? ""}')
                    ),

                  ],
                ),

          )

              .toList())

),
   ),

    );

}

</code>```

The issue is that values are there and Flutter says is null when the output is in the console.

```<code>===================
Launching lib/main.dart on iPhone 11 Pro Max in debug mode...
Running Xcode build...
Xcode build done.                                           17.9s
Waiting for iPhone 11 Pro Max to report its views...
Debug service listening on ws://127.0.0.1:57768/yyPV-GQyldg=/ws
Syncing files to device iPhone 11 Pro Max...
flutter: {"PGX":{"stats":{"week52change":-0.025465,"week52high":15.28,"week52low":9.71,"marketcap":4595628000,"employees":null,"day200MovingAvg":14.55,"day50MovingAvg":13.39,"float":null,"avg10Volume":2336007.5,"avg30Volume":2648722.77,"ttmEPS":null,"ttmDividendRate":0.8361,"companyName":"Invesco Preferred ETF","sharesOutstanding":324550000,"maxChangePercent":-0.3042,"year5ChangePercent":-0.0367,"year2ChangePercent":-0.0167,"year1ChangePercent":-0.025465,"ytdChangePercent":-0.064111,"month6ChangePercent":-0.045822,"month3ChangePercent":-0.052209,"month1ChangePercent":0.019438,"day30ChangePercent":0.028322,"day5ChangePercent":0.015054,"nextDividendDate":"2020-05-18","dividendYield":0.059046610169491524,"nextEarningsDate":null,"exDividendDate":"2020-05-18","peRatio":null,"beta":0.5684535732534528},"quote":{"symbol":"PGX","companyName":"Invesco Preferred ETF","primaryExchange":"NYSE Arca","calculationPrice":"close","open":14.11,"openTime":1590154200191,"openSource":"official","close":14.16,"closeT<…>
flutter: 2020","latestUpdate":1590177600180,"latestVolume":4747,"iexRealtimePrice":null,"iexRealtimeSize":null,"iexLastUpdated":null,"delayedPrice":11.245,"delayedPriceTime":1590192000004,"oddLotDelayedPrice":11.193,"oddLotDelayedPriceTime":1590173727614,"extendedPrice":11.245,"extendedChange":0.005,"extendedChangePercent":0.00044,"extendedPriceTime":1590186600005,"previousClose":11.16,"previousVolume":8688,"change":0.08,"changePercent":0.00717,"volume":4747,"iexMarketPercent":null,"iexVolume":null,"avgTotalVolume":12262,"iexBidPrice":null,"iexBidSize":null,"iexAskPrice":null,"iexAskSize":null,"iexOpen":null,"iexOpenTime":null,"iexClose":11.04,"iexCloseTime":1589988569572,"marketCap":35406000,"peRatio":null,"week52High":16.6,"week52Low":8.47,"ytdChange":-0.300712,"lastTradeTime":1590177600181,"isUSMarketOpen":false}},"IPFF":{"stats":{"week52change":-0.23383400000000001,"week52high":15.71,"week52low":8.38,"marketcap":28738500,"employees":null,"day200MovingAvg":13.89,"day50MovingAvg":10.96,"float<…>
flutter: 95
flutter: {"PGX":{"stats":{"week52change":-0.025465,"week52high":15.28,"week52low":9.71,"marketcap":4595628000,"employees":null,"day200MovingAvg":14.55,"day50MovingAvg":13.39,"float":null,"avg10Volume":2336007.5,"avg30Volume":2648722.77,"ttmEPS":null,"ttmDividendRate":0.8361,"companyName":"Invesco Preferred ETF","sharesOutstanding":324550000,"maxChangePercent":-0.3042,"year5ChangePercent":-0.0367,"year2ChangePercent":-0.0167,"year1ChangePercent":-0.025465,"ytdChangePercent":-0.064111,"month6ChangePercent":-0.045822,"month3ChangePercent":-0.052209,"month1ChangePercent":0.019438,"day30ChangePercent":0.028322,"day5ChangePercent":0.015054,"nextDividendDate":"2020-05-18","dividendYield":0.059046610169491524,"nextEarningsDate":null,"exDividendDate":"2020-05-18","peRatio":null,"beta":0.5684535732534528},"**quote":{"symbol":"PGX","companyName":"Invesco Preferred ETF","primaryExchange":"NYSE Arca","calculationPrice":"close","open":14.11,"openTime":1590154200191,"openSource":"official","close":14.16,"closeT<…>
flutter: 2020","latestUpdate":1590177600180,"latestVolume":4747,"iexRealtimePrice":null,"iexRealtimeSize":null,"iexLastUpdated":null,"delayedPrice":11.245,"delayedPriceTime":1590192000004,"oddLotDelayedPrice":11.193,"oddLotDelayedPriceTime":1590173727614,"extendedPrice":11.245,"extendedChange":0.005,"extendedChangePercent":0.00044,"extendedPriceTime":1590186600005,"previousClose":11.16,"previousVolume":8688,"change":0.08,"changePercent":0.00717,"volume":4747,"iexMarketPercent":null,"iexVolume":null,"avgTotalVolume":12262,"iexBidPrice":null,"iexBidSize":null,"iexAskPrice":null,"iexAskSize":null,"iexOpen":null,"iexOpenTime":null,"iexClose":11.04,"iexCloseTime":1589988569572,"marketCap":35406000,"peRatio":null,"week52High":16.6,"week52Low":8.47,"ytdChange":-0.300712,"lastTradeTime":1590177600181,"isUSMarketOpen":false}},"IPFF":{"stats":{"week52change":-0.23383400000000001,"week52high":15.71,"week52low":8.38,"marketcap":28738500,"employees":null,"day200MovingAvg":13.89,"day50MovingAvg":10.96,"float<…>
flutter: 95**

════════ Exception caught by widgets library ═══════════════════════════════════════════════════════
The following NoSuchMethodError was thrown building photorealist(dirty, state: _PhotosListState#8eef8):
The method '[]' was called on null.
Receiver: null
Tried calling: []("iexClose")
</code>```

=============
Column Price is empty.
If I try to call  "delayedPrice":11.552," or "iexClose":11.04. it does not print in the app.

答:
I have gone through the token you shared and tested the code you shared.

you are calling the array in a wrong way from where the error is being raised.

I tried it and output is being shown without any error

```<code>  DataCell(
      Text('${photo.data["quote"]["stats"]["latestPrice"] ?? ""}')   //❌
      //REPLACE   ["quote"]["stats"]    with   ["stats"]
      Text('${photo.data["stats"]["latestPrice"] ?? ""}')   //✔
  ),
</code>```

OUTPUT:

WORKING CODE

```<code>import 'dart:async';
import 'dart:convert';
import 'package:flutter/foundation.dart';
import 'package:flutter/material.dart';
import 'package:http/http.dart' as http;
//import 'package:flutter_web/material.dart';
//import 'package:flutter/main.dart' as app;
//import 'package:flutter_web_ui/ui.dart' as ui;

Future<List<Photo>> fetchPhotos(http.Client client) async {
  final response =
      // await client.get('https://cloud.iexapis.com/stable/stock/market/batch?symbols=CNPF,ICOL,PGX,MORL,EMLC,GYLD,IPFF,SMB,YDIV,YYY,PEY,IEI,TLH,DIA,BSJF,MONY&types=quote,stats&token=');

      await client.get(
          'https://cloud.iexapis.com/stable/stock/market/batch?symbols=PGX,MORL,EMLC,GYLD,IPFF,SMB,YDIV,YYY,PEY,PGF,SNLN,DVHL,AUNZ,MDIV,PHB,PFXF,FPE,TIPX,MLN,SDIV,SIPE,SEMF,EMAG,IEMF,EU,BSCM,LEMF,BSCL,BSCK,BSCJ,BSCI,DSUM,BSCF,KBWD,PCEF,BSCG,BSCH,HYEM,ITM,BKLN,PGHY,IBCE,PZT,IJNK,XMPT,THHY,QYLD,IHY,IYLD,SHYD,PRB,DVHI,GOVT,BSJJ,BSJK,RIGS,INY,IBCD,PZA,PFIG,IBCC,BSCE,PVI,MBG,CXA,FWDB,DIV,CHLC,BSJI,FLTR,PWZ,MUAH,IBDD,PICB,IBDB,BSJH,RVNU,BSJG,IBDC,XOVR,BSJE,EBND,PCY,BWX,SJNK,DES,ANGL,BAB,SST,EMCD,CBND,HYD,PLW,SCPB,FLRN,BWZ,KBWY,ELD,IBND,AOK,RWXL,ITR,DON,DGRS,JNK,TZD,EPU,PFF,HYLD,GGOV,ULST,SPHD,HFIN,DGRW,LWC,ALD,LEMB,AGND,ITIP,EMHY,QLTC,TFI,SRLN,HYLS,SHM,GNMA,CEMB,IGOV,MINC,AGZD,FTSL,SPLV,GHYG,ISTB,ENGN,CMBS,GTIP,SCHO,SMMU,QLTA,SCHZ,HSPX,TUZ,GIY,GSY,TFLO,VMBS,CWB,FLOT,STPZ,MUAC,SCHR,BABZ,HYXU,GMTB,MUNI,WIP,MUAD,QLTB,SCHP,GMMB,MUAE,IPE,NYF,BNDX,MUAF,HYMB,TIPZ,CMF,LAG,ILTB,ITE,CLY,VGSH,VGIT,LTPZ,BABS,HYHG,EMCB,DVYL,VGLT,DHS,TLO,VWOB,IGHG,GLCB,RAVI,VCSH,BSV,BND,BIV,ISHG,SDYL,VCIT,SHY,BLV,HYG,VCLT,TENZ,DTN,BIL,AUD,DLN,DTD,FTSD,CAD,STIP,CORP,HYS,HOLD,TRSY,IEF,IBDA,COBO,BOND,MINT,MBB,CSJ,AGG,SUB,EMB,CIU,GVI,MUB,GBF,TIP,AGZ,CFT,LQD,TLT,IEI,TLH,DIA,BSJF,MONY&types=stats,quote&token=token_was_added_here');
  //https://cloud.iexapis.com/v1/stock/market/batch?&types=price&symbols=aapl,fb,tsla&token=YOUR_TOKEN_HERE
  //https://cloud.iexapis.com/v1/stock/market/batch?&types=quote&symbols=aapl,fb,tsla&token=
  // Use the compute function to run parsePhotos in a separate isolate.
  //https://cloud.iexapis.com/stable/stock/market/batch?symbols=CNPF,ICOL,PGX,MORL,EMLC,GYLD,IPFF,SMB,YDIV,YYY,PEY,PGF,SNLN,DVHL,AUNZ,MDIV,PHB,PFXF,FPE,TIPX,MLN,SDIV,SIPE,SEMF,EMAG,IEMF,EU,BSCM,LEMF,BSCL,BSCK,BSCJ,BSCI,DSUM,BSCF,KBWD,PCEF,BSCG,BSCH,HYEM,ITM,BKLN,PGHY,IBCE,PZT,IJNK,XMPT,THHY,QYLD,IHY,IYLD,SHYD,PRB,DVHI,GOVT,BSJJ,BSJK,RIGS,INY,IBCD,PZA,PFIG,IBCC,BSCE,PVI,MBG,CXA,FWDB,DIV,CHLC,BSJI,FLTR,PWZ,MUAH,IBDD,PICB,IBDB,BSJH,RVNU,BSJG,IBDC,XOVR,BSJE,EBND,PCY,BWX,SJNK,DES,ANGL,BAB,SST,EMCD,CBND,HYD,PLW,SCPB,FLRN,BWZ,KBWY,ELD,IBND,AOK,RWXL,ITR,DON,DGRS,JNK,TZD,EPU,PFF,HYLD,GGOV,ULST,SPHD,HFIN,DGRW,LWC,ALD,LEMB,AGND,ITIP,EMHY,QLTC,TFI,SRLN,HYLS,SHM,GNMA,CEMB,IGOV,MINC,AGZD,FTSL,SPLV,GHYG,ISTB,ENGN,CMBS,GTIP,SCHO,SMMU,QLTA,SCHZ,HSPX,TUZ,GIY,GSY,TFLO,VMBS,CWB,FLOT,STPZ,MUAC,SCHR,BABZ,HYXU,GMTB,MUNI,WIP,MUAD,QLTB,SCHP,GMMB,MUAE,IPE,NYF,BNDX,MUAF,HYMB,TIPZ,CMF,LAG,ILTB,ITE,CLY,VGSH,VGIT,LTPZ,BABS,HYHG,EMCB,DVYL,VGLT,DHS,TLO,VWOB,IGHG,GLCB,RAVI,VCSH,BSV,BND,BIV,ISHG,SDYL,VCIT,SHY,BLV,HYG,VCLT,TENZ,DTN,BIL,AUD,DLN,DTD,FTSD,CAD,STIP,CORP,HYS,HOLD,TRSY,IEF,IBDA,COBO,BOND,MINT,MBB,CSJ,AGG,SUB,EMB,CIU,GVI,MUB,GBF,TIP,AGZ,CFT,LQD,TLT,IEI,TLH,DIA,BSJF,MONY&types=quote,stats&token=
  return compute(parsePhotos, response.body);
}

// A function that converts a response body into a List<Photo>.
List<Photo> parsePhotos(String responseBody) {
  //final parsed = json.decode(responseBody).cast<Map<String,dynamic>>();

  //return parsed.map<Photo>((json) => Photo.fromJson(json)).toList();

  dynamic Obj = json.decode(responseBody);
  debugPrint(responseBody, wrapWidth: 8192);
  print(Obj.length);

  List<Photo> photoList = [];
  Obj.forEach((k, v) => photoList.add(Photo(k, v)));

  return photoList;
}

class Photo {
  String symbol;
  //String companyName;
  dynamic data;
  // dynamic iexClose;
  // dynamic quote;
  Photo(this.symbol, this.data);
}

void main() => runApp(MyApp());

class MyApp extends StatelessWidget {
  @override
  Widget build(BuildContext context) {
    final appTitle = 'Monthly Paying ETFs';

    return MaterialApp(
      title: appTitle,
      home: MyHomePage(title: appTitle),
    );
  }
}

class MyHomePage extends StatelessWidget {
  final String title;

  MyHomePage({Key key, this.title}) : super(key: key);

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      drawer: Drawer(
        // Add a ListView to the drawer. This ensures the user can scroll
        // through the options in the drawer if there isn't enough vertical
        // space to fit everything.
        child: ListView(
          // Important: Remove any padding from the ListView.
          padding: EdgeInsets.zero,
          children: <Widget>[
            DrawerHeader(
              child: Text('About'),
              decoration: BoxDecoration(
                color: Colors.grey,
              ),
            ),
            ListTile(
              title: Text(
                  'This app is to provide the list of ETF s that pay monthly dividend'),
              onTap: () {
                // Update the state of the app
                // ...
                // Then close the drawer
                Navigator.pop(context);
              },
            ),
            ListTile(
              title: Text('for questions or feedback please contact me at  '),
              onTap: () {
                // Update the state of the app
                // ...
                // Then close the drawer
                Navigator.pop(context);
              },
            ),
          ],
        ),
      ),
      appBar: AppBar(
        title: Text(title),
        backgroundColor: Colors.greenAccent,
      ),
      body: FutureBuilder<List<Photo>>(
        future: fetchPhotos(http.Client()),
        builder: (context, snapshot) {
          if (snapshot.hasError) print(snapshot.error);

          return snapshot.hasData
              ? PhotosList(photos: snapshot.data)
              : Center(child: CircularProgressIndicator());
        },
      ),
    );
  }
}

class PhotosList extends StatefulWidget {
  final List<Photo> photos;

  PhotosList({Key key, this.photos})
      : assert(photos != null),
        super(key: key);

  @override
  _PhotosListState createState() => _PhotosListState();
}

class _PhotosListState extends State<PhotosList> {
  @override
  Widget build(BuildContext context) {
    return bodyData();
  }

  SingleChildScrollView bodyData() => SingleChildScrollView(
        scrollDirection: Axis.vertical,

        // padding: EdgeInsets.all(0.0),
        //padding: const EdgeInsets.symmetric(
        //  padding: new EdgeInsets.fromLTRB(0.001, .001, 1.0, 1.0),
        padding: EdgeInsets.all(0.05),

        // child: FittedBox(fit:BoxFit.contain,

        child: SingleChildScrollView(
          scrollDirection: Axis.horizontal,
          child: FittedBox(
              fit: BoxFit.fitHeight,
              child: DataTable(
                  // sortColumnIndex: 1,
                  //  sortAscending: true,

                  columns: <DataColumn>[
                    DataColumn(
                      label: Text("Company"),
                      onSort: (_, __) {
                        setState(() {
                          widget.photos.sort((a, b) => a.data["quote"]
                                  ["companyName"]
                              .compareTo(b.data["quote"]["companyName"]));
                        });
                      },
                    ),
                    DataColumn(
                      label: Text("ttmDivRate"),
                      numeric: true,
                      onSort: (_, __) {
                        setState(() {
                          widget.photos.sort((a, b) => a.data["quote"]
                                  ["ttmDividendRate"]
                              .compareTo(b.data["quote"]["ttmDividendRate"]));
                        });
                      },
                    ),
                    DataColumn(
                      label: Text("Price"),
                      numeric: true,
                      onSort: (_, __) {
                        setState(() {
                          widget.photos.sort((a, b) => a.data["quote"]
                                  ["latestPrice"]
                              .compareTo(b.data["quote"]["latestPrice"]));
                        });
                      },
                    ),
                  ],
                  rows: widget.photos
                      .map(
                        (photo) => DataRow(
                          cells: [
                            DataCell(
                              Text(
                                  '${photo.data["stats"]["companyName"] ?? ""}'),
                            ),

                            DataCell(
                              Text(
                                  '${photo.data["stats"]["ttmDividendRate"] ?? ""}'),
                            ),
//["quote"]["stats"]
                            DataCell(Text(
                                '${photo.data["stats"]["latestPrice"] ?? ""}')),
                          ],
                        ),
                      )
                      .toList())),
        ),
      );
}
</code>```

Body:
To isolate the issue,  I created a new code, a new program, and a new machine.
Same results, it just doesn't want to print any value that has  is after  "quotes and has "anythingPrice"

Again I can give you a temp key and you can fully run the code. on your side.

Please let me know.

import 'dart:async';
import 'dart:convert';
import 'package:flutter/foundation.dart';
import 'package:flutter/material.dart';
import 'package:http/http.dart' as http;
//import 'package:flutter_web/material.dart';
//import 'package:flutter/main.dart' as app;
//import 'package:flutter_web_ui/ui.dart' as ui;

Future<List<Photo>> fetchPhotos(http.Client client) async {
  final response =
 // await client.get('https://cloud.iexapis.com/stable/stock/market/batch?symbols=CNPF,ICOL,PGX,MORL,EMLC,GYLD,IPFF,SMB,YDIV,YYY,PEY,IEI,TLH,DIA,BSJF,MONY&types=quote,stats&token=');

  await client.get('https://cloud.iexapis.com/stable/stock/market/batch?symbols=PGX,MORL,EMLC,GYLD,IPFF,SMB,YDIV,YYY,PEY,PGF,SNLN,DVHL,AUNZ,MDIV,PHB,PFXF,FPE,TIPX,MLN,SDIV,SIPE,SEMF,EMAG,IEMF,EU,BSCM,LEMF,BSCL,BSCK,BSCJ,BSCI,DSUM,BSCF,KBWD,PCEF,BSCG,BSCH,HYEM,ITM,BKLN,PGHY,IBCE,PZT,IJNK,XMPT,THHY,QYLD,IHY,IYLD,SHYD,PRB,DVHI,GOVT,BSJJ,BSJK,RIGS,INY,IBCD,PZA,PFIG,IBCC,BSCE,PVI,MBG,CXA,FWDB,DIV,CHLC,BSJI,FLTR,PWZ,MUAH,IBDD,PICB,IBDB,BSJH,RVNU,BSJG,IBDC,XOVR,BSJE,EBND,PCY,BWX,SJNK,DES,ANGL,BAB,SST,EMCD,CBND,HYD,PLW,SCPB,FLRN,BWZ,KBWY,ELD,IBND,AOK,RWXL,ITR,DON,DGRS,JNK,TZD,EPU,PFF,HYLD,GGOV,ULST,SPHD,HFIN,DGRW,LWC,ALD,LEMB,AGND,ITIP,EMHY,QLTC,TFI,SRLN,HYLS,SHM,GNMA,CEMB,IGOV,MINC,AGZD,FTSL,SPLV,GHYG,ISTB,ENGN,CMBS,GTIP,SCHO,SMMU,QLTA,SCHZ,HSPX,TUZ,GIY,GSY,TFLO,VMBS,CWB,FLOT,STPZ,MUAC,SCHR,BABZ,HYXU,GMTB,MUNI,WIP,MUAD,QLTB,SCHP,GMMB,MUAE,IPE,NYF,BNDX,MUAF,HYMB,TIPZ,CMF,LAG,ILTB,ITE,CLY,VGSH,VGIT,LTPZ,BABS,HYHG,EMCB,DVYL,VGLT,DHS,TLO,VWOB,IGHG,GLCB,RAVI,VCSH,BSV,BND,BIV,ISHG,SDYL,VCIT,SHY,BLV,HYG,VCLT,TENZ,DTN,BIL,AUD,DLN,DTD,FTSD,CAD,STIP,CORP,HYS,HOLD,TRSY,IEF,IBDA,COBO,BOND,MINT,MBB,CSJ,AGG,SUB,EMB,CIU,GVI,MUB,GBF,TIP,AGZ,CFT,LQD,TLT,IEI,TLH,DIA,BSJF,MONY&types=stats,quote&token=');
  //https://cloud.iexapis.com/v1/stock/market/batch?&types=price&symbols=aapl,fb,tsla&token=YOUR_TOKEN_HERE
  //https://cloud.iexapis.com/v1/stock/market/batch?&types=quote&symbols=aapl,fb,tsla&token=
  // Use the compute function to run parsePhotos in a separate isolate.
  //https://cloud.iexapis.com/stable/stock/market/batch?symbols=CNPF,ICOL,PGX,MORL,EMLC,GYLD,IPFF,SMB,YDIV,YYY,PEY,PGF,SNLN,DVHL,AUNZ,MDIV,PHB,PFXF,FPE,TIPX,MLN,SDIV,SIPE,SEMF,EMAG,IEMF,EU,BSCM,LEMF,BSCL,BSCK,BSCJ,BSCI,DSUM,BSCF,KBWD,PCEF,BSCG,BSCH,HYEM,ITM,BKLN,PGHY,IBCE,PZT,IJNK,XMPT,THHY,QYLD,IHY,IYLD,SHYD,PRB,DVHI,GOVT,BSJJ,BSJK,RIGS,INY,IBCD,PZA,PFIG,IBCC,BSCE,PVI,MBG,CXA,FWDB,DIV,CHLC,BSJI,FLTR,PWZ,MUAH,IBDD,PICB,IBDB,BSJH,RVNU,BSJG,IBDC,XOVR,BSJE,EBND,PCY,BWX,SJNK,DES,ANGL,BAB,SST,EMCD,CBND,HYD,PLW,SCPB,FLRN,BWZ,KBWY,ELD,IBND,AOK,RWXL,ITR,DON,DGRS,JNK,TZD,EPU,PFF,HYLD,GGOV,ULST,SPHD,HFIN,DGRW,LWC,ALD,LEMB,AGND,ITIP,EMHY,QLTC,TFI,SRLN,HYLS,SHM,GNMA,CEMB,IGOV,MINC,AGZD,FTSL,SPLV,GHYG,ISTB,ENGN,CMBS,GTIP,SCHO,SMMU,QLTA,SCHZ,HSPX,TUZ,GIY,GSY,TFLO,VMBS,CWB,FLOT,STPZ,MUAC,SCHR,BABZ,HYXU,GMTB,MUNI,WIP,MUAD,QLTB,SCHP,GMMB,MUAE,IPE,NYF,BNDX,MUAF,HYMB,TIPZ,CMF,LAG,ILTB,ITE,CLY,VGSH,VGIT,LTPZ,BABS,HYHG,EMCB,DVYL,VGLT,DHS,TLO,VWOB,IGHG,GLCB,RAVI,VCSH,BSV,BND,BIV,ISHG,SDYL,VCIT,SHY,BLV,HYG,VCLT,TENZ,DTN,BIL,AUD,DLN,DTD,FTSD,CAD,STIP,CORP,HYS,HOLD,TRSY,IEF,IBDA,COBO,BOND,MINT,MBB,CSJ,AGG,SUB,EMB,CIU,GVI,MUB,GBF,TIP,AGZ,CFT,LQD,TLT,IEI,TLH,DIA,BSJF,MONY&types=quote,stats&token=
  return compute(parsePhotos, response.body);
}

// A function that converts a response body into a List<Photo>.
List<Photo> parsePhotos(String responseBody) {
  //final parsed = json.decode(responseBody).cast<Map<String,dynamic>>();

  //return parsed.map<Photo>((json) => Photo.fromJson(json)).toList();

  dynamic Obj = json.decode(responseBody);
  debugPrint(responseBody, wrapWidth: 8192);
  print(Obj.length);

  List<Photo> photoList = [];
  Obj.forEach((k, v) => photoList.add(Photo(k,v)));

  return photoList;
}

class Photo {
  String symbol;
  //String companyName;
  dynamic data;
 // dynamic iexClose;
 // dynamic quote;
  Photo(this.symbol ,this.data);

}

void main() => runApp(MyApp());

class MyApp extends StatelessWidget {
  @override

  Widget build(BuildContext context) {

    final appTitle = 'Monthly Paying ETFs';

    return MaterialApp(
      title: appTitle,
      home: MyHomePage(title: appTitle),

    );
  }
}

class MyHomePage extends StatelessWidget {

  final String title;

  MyHomePage({Key key, this.title}) : super(key: key);

  @override
  Widget build(BuildContext context) {

    return Scaffold(

        drawer: Drawer(
          // Add a ListView to the drawer. This ensures the user can scroll
          // through the options in the drawer if there isn't enough vertical
          // space to fit everything.
          child: ListView(
            // Important: Remove any padding from the ListView.
            padding: EdgeInsets.zero,
            children: <Widget>[
              DrawerHeader(
                child: Text('About'),
                decoration: BoxDecoration(
                  color: Colors.grey,
                ),
              ),
              ListTile(
                title: Text('This app is to provide the list of ETF s that pay monthly dividend'),

                onTap: () {
                  // Update the state of the app
                  // ...
                  // Then close the drawer
                  Navigator.pop(context);
                },

              ),
              ListTile(
                title: Text('for questions or feedback please contact me at  '),

                onTap: () {
                  // Update the state of the app
                  // ...
                  // Then close the drawer
                  Navigator.pop(context);
                },

              ),

            ],
          ),
        ),

        appBar: AppBar(
        title: Text(title),
          backgroundColor: Colors.greenAccent,
      ),

      body: FutureBuilder<List<Photo>>(

        future: fetchPhotos(http.Client()),

        builder: (context, snapshot) {
          if (snapshot.hasError) print(snapshot.error);

          return snapshot.hasData
              ? PhotosList(photos: snapshot.data)
              : Center(child: CircularProgressIndicator());

        },
      ),
    );
  }
}

class PhotosList extends StatefulWidget {
  final List photos;

  PhotosList({Key key, this.photos})
      : assert(photos != null),
        super(key: key);

  @override
  _PhotosListState createState() => _PhotosListState();
}

class _PhotosListState extends State<PhotosList> {
  @override

  Widget build(BuildContext context) {

    return bodyData();

  }

  SingleChildScrollView bodyData() =>

    SingleChildScrollView(
  scrollDirection: Axis.vertical,

      // padding: EdgeInsets.all(0.0),
       //padding: const EdgeInsets.symmetric(
       //  padding: new EdgeInsets.fromLTRB(0.001, .001, 1.0, 1.0),
       padding: EdgeInsets.all(0.05),

 // child: FittedBox(fit:BoxFit.contain,

  child:SingleChildScrollView(
    scrollDirection: Axis.horizontal,

  child:
  FittedBox(
  fit: BoxFit.fitHeight,
  child: DataTable(
         // sortColumnIndex: 1,
        //  sortAscending: true,

          columns: <DataColumn>[
            DataColumn(
              label: Text("Company"),
              onSort: (_, __) {
                setState(() {
                  widget.photos.sort((a, b) =>
                      a.data["quote"]["companyName"]
                          .compareTo(b.data["quote"]["companyName"]));
                });
              },
            ),

            DataColumn(
              label: Text("ttmDivRate"),
              numeric: true,
              onSort:   (_,__) {
                setState(() {
                  widget.photos.sort((a, b) =>
                      a.data["quote"]["ttmDividendRate"]
                          .compareTo(b.data["quote"]["ttmDividendRate"]));
                });
              },
            ),

            DataColumn(
              label: Text("Price"),
              numeric: true,
              onSort: (_, __) {
                setState(() {
                  widget.photos.sort((a, b) => a.data ["quote"]["latestPrice"]
                      .compareTo(b.data["quote"]["latestPrice"]));
                });
              },
            ),

          ],

          rows: widget.photos

              .map(
                (photo) =>

                DataRow(
                  cells: [

                    DataCell(
                      Text('${photo.data["stats"]["companyName"] ?? ""}'),
                    ),

                    DataCell(
                      Text('${photo.data["stats"]["ttmDividendRate"] ?? ""}'),
                   ),
//["quote"]["stats"]
                    DataCell(
                      Text('${photo.data["quote"]["stats"]["latestPrice"] ?? ""}')
                    ),

                  ],
                ),

          )

              .toList())

),
   ),

    );

}

The issue is that values are there and Flutter says is null when the output is in the console.

===================
Launching lib/main.dart on iPhone 11 Pro Max in debug mode...
Running Xcode build...
Xcode build done.                                           17.9s
Waiting for iPhone 11 Pro Max to report its views...
Debug service listening on ws://127.0.0.1:57768/yyPV-GQyldg=/ws
Syncing files to device iPhone 11 Pro Max...
flutter: {"PGX":{"stats":{"week52change":-0.025465,"week52high":15.28,"week52low":9.71,"marketcap":4595628000,"employees":null,"day200MovingAvg":14.55,"day50MovingAvg":13.39,"float":null,"avg10Volume":2336007.5,"avg30Volume":2648722.77,"ttmEPS":null,"ttmDividendRate":0.8361,"companyName":"Invesco Preferred ETF","sharesOutstanding":324550000,"maxChangePercent":-0.3042,"year5ChangePercent":-0.0367,"year2ChangePercent":-0.0167,"year1ChangePercent":-0.025465,"ytdChangePercent":-0.064111,"month6ChangePercent":-0.045822,"month3ChangePercent":-0.052209,"month1ChangePercent":0.019438,"day30ChangePercent":0.028322,"day5ChangePercent":0.015054,"nextDividendDate":"2020-05-18","dividendYield":0.059046610169491524,"nextEarningsDate":null,"exDividendDate":"2020-05-18","peRatio":null,"beta":0.5684535732534528},"quote":{"symbol":"PGX","companyName":"Invesco Preferred ETF","primaryExchange":"NYSE Arca","calculationPrice":"close","open":14.11,"openTime":1590154200191,"openSource":"official","close":14.16,"closeT<…>
flutter: 2020","latestUpdate":1590177600180,"latestVolume":4747,"iexRealtimePrice":null,"iexRealtimeSize":null,"iexLastUpdated":null,"delayedPrice":11.245,"delayedPriceTime":1590192000004,"oddLotDelayedPrice":11.193,"oddLotDelayedPriceTime":1590173727614,"extendedPrice":11.245,"extendedChange":0.005,"extendedChangePercent":0.00044,"extendedPriceTime":1590186600005,"previousClose":11.16,"previousVolume":8688,"change":0.08,"changePercent":0.00717,"volume":4747,"iexMarketPercent":null,"iexVolume":null,"avgTotalVolume":12262,"iexBidPrice":null,"iexBidSize":null,"iexAskPrice":null,"iexAskSize":null,"iexOpen":null,"iexOpenTime":null,"iexClose":11.04,"iexCloseTime":1589988569572,"marketCap":35406000,"peRatio":null,"week52High":16.6,"week52Low":8.47,"ytdChange":-0.300712,"lastTradeTime":1590177600181,"isUSMarketOpen":false}},"IPFF":{"stats":{"week52change":-0.23383400000000001,"week52high":15.71,"week52low":8.38,"marketcap":28738500,"employees":null,"day200MovingAvg":13.89,"day50MovingAvg":10.96,"float<…>
flutter: 95
flutter: {"PGX":{"stats":{"week52change":-0.025465,"week52high":15.28,"week52low":9.71,"marketcap":4595628000,"employees":null,"day200MovingAvg":14.55,"day50MovingAvg":13.39,"float":null,"avg10Volume":2336007.5,"avg30Volume":2648722.77,"ttmEPS":null,"ttmDividendRate":0.8361,"companyName":"Invesco Preferred ETF","sharesOutstanding":324550000,"maxChangePercent":-0.3042,"year5ChangePercent":-0.0367,"year2ChangePercent":-0.0167,"year1ChangePercent":-0.025465,"ytdChangePercent":-0.064111,"month6ChangePercent":-0.045822,"month3ChangePercent":-0.052209,"month1ChangePercent":0.019438,"day30ChangePercent":0.028322,"day5ChangePercent":0.015054,"nextDividendDate":"2020-05-18","dividendYield":0.059046610169491524,"nextEarningsDate":null,"exDividendDate":"2020-05-18","peRatio":null,"beta":0.5684535732534528},"**quote":{"symbol":"PGX","companyName":"Invesco Preferred ETF","primaryExchange":"NYSE Arca","calculationPrice":"close","open":14.11,"openTime":1590154200191,"openSource":"official","close":14.16,"closeT<…>
flutter: 2020","latestUpdate":1590177600180,"latestVolume":4747,"iexRealtimePrice":null,"iexRealtimeSize":null,"iexLastUpdated":null,"delayedPrice":11.245,"delayedPriceTime":1590192000004,"oddLotDelayedPrice":11.193,"oddLotDelayedPriceTime":1590173727614,"extendedPrice":11.245,"extendedChange":0.005,"extendedChangePercent":0.00044,"extendedPriceTime":1590186600005,"previousClose":11.16,"previousVolume":8688,"change":0.08,"changePercent":0.00717,"volume":4747,"iexMarketPercent":null,"iexVolume":null,"avgTotalVolume":12262,"iexBidPrice":null,"iexBidSize":null,"iexAskPrice":null,"iexAskSize":null,"iexOpen":null,"iexOpenTime":null,"iexClose":11.04,"iexCloseTime":1589988569572,"marketCap":35406000,"peRatio":null,"week52High":16.6,"week52Low":8.47,"ytdChange":-0.300712,"lastTradeTime":1590177600181,"isUSMarketOpen":false}},"IPFF":{"stats":{"week52change":-0.23383400000000001,"week52high":15.71,"week52low":8.38,"marketcap":28738500,"employees":null,"day200MovingAvg":13.89,"day50MovingAvg":10.96,"float<…>
flutter: 95**

════════ Exception caught by widgets library ═══════════════════════════════════════════════════════
The following NoSuchMethodError was thrown building photorealist(dirty, state: _PhotosListState#8eef8):
The method '[]' was called on null.
Receiver: null
Tried calling: []("iexClose")

=============
Column Price is empty.
If I try to call  "delayedPrice":11.552," or "iexClose":11.04. it does not print in the app.

Body:
I have gone through the token you shared and tested the code you shared.

you are calling the array in a wrong way from where the error is being raised.

I tried it and output is being shown without any error

  DataCell(
      Text('${photo.data["quote"]["stats"]["latestPrice"] ?? ""}')   //❌
      //REPLACE   ["quote"]["stats"]    with   ["stats"]
      Text('${photo.data["stats"]["latestPrice"] ?? ""}')   //✔
  ),

OUTPUT:

WORKING CODE

import 'dart:async';
import 'dart:convert';
import 'package:flutter/foundation.dart';
import 'package:flutter/material.dart';
import 'package:http/http.dart' as http;
//import 'package:flutter_web/material.dart';
//import 'package:flutter/main.dart' as app;
//import 'package:flutter_web_ui/ui.dart' as ui;

Future<List<Photo>> fetchPhotos(http.Client client) async {
  final response =
      // await client.get('https://cloud.iexapis.com/stable/stock/market/batch?symbols=CNPF,ICOL,PGX,MORL,EMLC,GYLD,IPFF,SMB,YDIV,YYY,PEY,IEI,TLH,DIA,BSJF,MONY&types=quote,stats&token=');

      await client.get(
          'https://cloud.iexapis.com/stable/stock/market/batch?symbols=PGX,MORL,EMLC,GYLD,IPFF,SMB,YDIV,YYY,PEY,PGF,SNLN,DVHL,AUNZ,MDIV,PHB,PFXF,FPE,TIPX,MLN,SDIV,SIPE,SEMF,EMAG,IEMF,EU,BSCM,LEMF,BSCL,BSCK,BSCJ,BSCI,DSUM,BSCF,KBWD,PCEF,BSCG,BSCH,HYEM,ITM,BKLN,PGHY,IBCE,PZT,IJNK,XMPT,THHY,QYLD,IHY,IYLD,SHYD,PRB,DVHI,GOVT,BSJJ,BSJK,RIGS,INY,IBCD,PZA,PFIG,IBCC,BSCE,PVI,MBG,CXA,FWDB,DIV,CHLC,BSJI,FLTR,PWZ,MUAH,IBDD,PICB,IBDB,BSJH,RVNU,BSJG,IBDC,XOVR,BSJE,EBND,PCY,BWX,SJNK,DES,ANGL,BAB,SST,EMCD,CBND,HYD,PLW,SCPB,FLRN,BWZ,KBWY,ELD,IBND,AOK,RWXL,ITR,DON,DGRS,JNK,TZD,EPU,PFF,HYLD,GGOV,ULST,SPHD,HFIN,DGRW,LWC,ALD,LEMB,AGND,ITIP,EMHY,QLTC,TFI,SRLN,HYLS,SHM,GNMA,CEMB,IGOV,MINC,AGZD,FTSL,SPLV,GHYG,ISTB,ENGN,CMBS,GTIP,SCHO,SMMU,QLTA,SCHZ,HSPX,TUZ,GIY,GSY,TFLO,VMBS,CWB,FLOT,STPZ,MUAC,SCHR,BABZ,HYXU,GMTB,MUNI,WIP,MUAD,QLTB,SCHP,GMMB,MUAE,IPE,NYF,BNDX,MUAF,HYMB,TIPZ,CMF,LAG,ILTB,ITE,CLY,VGSH,VGIT,LTPZ,BABS,HYHG,EMCB,DVYL,VGLT,DHS,TLO,VWOB,IGHG,GLCB,RAVI,VCSH,BSV,BND,BIV,ISHG,SDYL,VCIT,SHY,BLV,HYG,VCLT,TENZ,DTN,BIL,AUD,DLN,DTD,FTSD,CAD,STIP,CORP,HYS,HOLD,TRSY,IEF,IBDA,COBO,BOND,MINT,MBB,CSJ,AGG,SUB,EMB,CIU,GVI,MUB,GBF,TIP,AGZ,CFT,LQD,TLT,IEI,TLH,DIA,BSJF,MONY&types=stats,quote&token=token_was_added_here');
  //https://cloud.iexapis.com/v1/stock/market/batch?&types=price&symbols=aapl,fb,tsla&token=YOUR_TOKEN_HERE
  //https://cloud.iexapis.com/v1/stock/market/batch?&types=quote&symbols=aapl,fb,tsla&token=
  // Use the compute function to run parsePhotos in a separate isolate.
  //https://cloud.iexapis.com/stable/stock/market/batch?symbols=CNPF,ICOL,PGX,MORL,EMLC,GYLD,IPFF,SMB,YDIV,YYY,PEY,PGF,SNLN,DVHL,AUNZ,MDIV,PHB,PFXF,FPE,TIPX,MLN,SDIV,SIPE,SEMF,EMAG,IEMF,EU,BSCM,LEMF,BSCL,BSCK,BSCJ,BSCI,DSUM,BSCF,KBWD,PCEF,BSCG,BSCH,HYEM,ITM,BKLN,PGHY,IBCE,PZT,IJNK,XMPT,THHY,QYLD,IHY,IYLD,SHYD,PRB,DVHI,GOVT,BSJJ,BSJK,RIGS,INY,IBCD,PZA,PFIG,IBCC,BSCE,PVI,MBG,CXA,FWDB,DIV,CHLC,BSJI,FLTR,PWZ,MUAH,IBDD,PICB,IBDB,BSJH,RVNU,BSJG,IBDC,XOVR,BSJE,EBND,PCY,BWX,SJNK,DES,ANGL,BAB,SST,EMCD,CBND,HYD,PLW,SCPB,FLRN,BWZ,KBWY,ELD,IBND,AOK,RWXL,ITR,DON,DGRS,JNK,TZD,EPU,PFF,HYLD,GGOV,ULST,SPHD,HFIN,DGRW,LWC,ALD,LEMB,AGND,ITIP,EMHY,QLTC,TFI,SRLN,HYLS,SHM,GNMA,CEMB,IGOV,MINC,AGZD,FTSL,SPLV,GHYG,ISTB,ENGN,CMBS,GTIP,SCHO,SMMU,QLTA,SCHZ,HSPX,TUZ,GIY,GSY,TFLO,VMBS,CWB,FLOT,STPZ,MUAC,SCHR,BABZ,HYXU,GMTB,MUNI,WIP,MUAD,QLTB,SCHP,GMMB,MUAE,IPE,NYF,BNDX,MUAF,HYMB,TIPZ,CMF,LAG,ILTB,ITE,CLY,VGSH,VGIT,LTPZ,BABS,HYHG,EMCB,DVYL,VGLT,DHS,TLO,VWOB,IGHG,GLCB,RAVI,VCSH,BSV,BND,BIV,ISHG,SDYL,VCIT,SHY,BLV,HYG,VCLT,TENZ,DTN,BIL,AUD,DLN,DTD,FTSD,CAD,STIP,CORP,HYS,HOLD,TRSY,IEF,IBDA,COBO,BOND,MINT,MBB,CSJ,AGG,SUB,EMB,CIU,GVI,MUB,GBF,TIP,AGZ,CFT,LQD,TLT,IEI,TLH,DIA,BSJF,MONY&types=quote,stats&token=
  return compute(parsePhotos, response.body);
}

// A function that converts a response body into a List<Photo>.
List<Photo> parsePhotos(String responseBody) {
  //final parsed = json.decode(responseBody).cast<Map<String,dynamic>>();

  //return parsed.map<Photo>((json) => Photo.fromJson(json)).toList();

  dynamic Obj = json.decode(responseBody);
  debugPrint(responseBody, wrapWidth: 8192);
  print(Obj.length);

  List<Photo> photoList = [];
  Obj.forEach((k, v) => photoList.add(Photo(k, v)));

  return photoList;
}

class Photo {
  String symbol;
  //String companyName;
  dynamic data;
  // dynamic iexClose;
  // dynamic quote;
  Photo(this.symbol, this.data);
}

void main() => runApp(MyApp());

class MyApp extends StatelessWidget {
  @override
  Widget build(BuildContext context) {
    final appTitle = 'Monthly Paying ETFs';

    return MaterialApp(
      title: appTitle,
      home: MyHomePage(title: appTitle),
    );
  }
}

class MyHomePage extends StatelessWidget {
  final String title;

  MyHomePage({Key key, this.title}) : super(key: key);

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      drawer: Drawer(
        // Add a ListView to the drawer. This ensures the user can scroll
        // through the options in the drawer if there isn't enough vertical
        // space to fit everything.
        child: ListView(
          // Important: Remove any padding from the ListView.
          padding: EdgeInsets.zero,
          children: <Widget>[
            DrawerHeader(
              child: Text('About'),
              decoration: BoxDecoration(
                color: Colors.grey,
              ),
            ),
            ListTile(
              title: Text(
                  'This app is to provide the list of ETF s that pay monthly dividend'),
              onTap: () {
                // Update the state of the app
                // ...
                // Then close the drawer
                Navigator.pop(context);
              },
            ),
            ListTile(
              title: Text('for questions or feedback please contact me at  '),
              onTap: () {
                // Update the state of the app
                // ...
                // Then close the drawer
                Navigator.pop(context);
              },
            ),
          ],
        ),
      ),
      appBar: AppBar(
        title: Text(title),
        backgroundColor: Colors.greenAccent,
      ),
      body: FutureBuilder<List<Photo>>(
        future: fetchPhotos(http.Client()),
        builder: (context, snapshot) {
          if (snapshot.hasError) print(snapshot.error);

          return snapshot.hasData
              ? PhotosList(photos: snapshot.data)
              : Center(child: CircularProgressIndicator());
        },
      ),
    );
  }
}

class PhotosList extends StatefulWidget {
  final List<Photo> photos;

  PhotosList({Key key, this.photos})
      : assert(photos != null),
        super(key: key);

  @override
  _PhotosListState createState() => _PhotosListState();
}

class _PhotosListState extends State<PhotosList> {
  @override
  Widget build(BuildContext context) {
    return bodyData();
  }

  SingleChildScrollView bodyData() => SingleChildScrollView(
        scrollDirection: Axis.vertical,

        // padding: EdgeInsets.all(0.0),
        //padding: const EdgeInsets.symmetric(
        //  padding: new EdgeInsets.fromLTRB(0.001, .001, 1.0, 1.0),
        padding: EdgeInsets.all(0.05),

        // child: FittedBox(fit:BoxFit.contain,

        child: SingleChildScrollView(
          scrollDirection: Axis.horizontal,
          child: FittedBox(
              fit: BoxFit.fitHeight,
              child: DataTable(
                  // sortColumnIndex: 1,
                  //  sortAscending: true,

                  columns: <DataColumn>[
                    DataColumn(
                      label: Text("Company"),
                      onSort: (_, __) {
                        setState(() {
                          widget.photos.sort((a, b) => a.data["quote"]
                                  ["companyName"]
                              .compareTo(b.data["quote"]["companyName"]));
                        });
                      },
                    ),
                    DataColumn(
                      label: Text("ttmDivRate"),
                      numeric: true,
                      onSort: (_, __) {
                        setState(() {
                          widget.photos.sort((a, b) => a.data["quote"]
                                  ["ttmDividendRate"]
                              .compareTo(b.data["quote"]["ttmDividendRate"]));
                        });
                      },
                    ),
                    DataColumn(
                      label: Text("Price"),
                      numeric: true,
                      onSort: (_, __) {
                        setState(() {
                          widget.photos.sort((a, b) => a.data["quote"]
                                  ["latestPrice"]
                              .compareTo(b.data["quote"]["latestPrice"]));
                        });
                      },
                    ),
                  ],
                  rows: widget.photos
                      .map(
                        (photo) => DataRow(
                          cells: [
                            DataCell(
                              Text(
                                  '${photo.data["stats"]["companyName"] ?? ""}'),
                            ),

                            DataCell(
                              Text(
                                  '${photo.data["stats"]["ttmDividendRate"] ?? ""}'),
                            ),
//["quote"]["stats"]
                            DataCell(Text(
                                '${photo.data["stats"]["latestPrice"] ?? ""}')),
                          ],
                        ),
                      )
                      .toList())),
        ),
      );
}


问:
How to convert column of strings into sequential numbers in Pandas?
I have a column in a df that is full of strings like like ["1120", "2230", "1120", "1234" ...]
where not every value in this column is unique.

I want to convert these strings into SEQUENTIAL NUMBERS from 0 to N where N is the number of unique values in that column so that I can make a scatterplot with this data. Simply changing the type of the column is not sufficient for this task. Any guidance is much appreciated. I tried using dummy variables, but don't really know where to start.

答:
Just use .unique() and .reset_index() to get a lookup table from the strings to sequential ID and then .join() the tables:

```<code>df = pd.DataFrame(["1120", "2230", "1120", "1234"], columns=["num"])
sequential = pd.Series(df["num"].unique()).reset_index().rename(columns={0: "num"})
df.merge(sequential, on="num")
</code>```

EDIT:

If you want to sort the number strings by their integer values first, you can add this line of code to sort (before you create the sequential Series):

```<code>df["num"] = df["num"].astype("int").sort_values().reset_index(drop=True)
</code>```

Body:
Just use .unique() and .reset_index() to get a lookup table from the strings to sequential ID and then .join() the tables:

df = pd.DataFrame(["1120", "2230", "1120", "1234"], columns=["num"])
sequential = pd.Series(df["num"].unique()).reset_index().rename(columns={0: "num"})
df.merge(sequential, on="num")

EDIT:

If you want to sort the number strings by their integer values first, you can add this line of code to sort (before you create the sequential Series):

df["num"] = df["num"].astype("int").sort_values().reset_index(drop=True)


问:
How to assign content their own height using grid?
https://codepen.io/daniel-brown/pen/QWjXpXw

I need this li, to have their own height and not someone else’s

Expected outcome

div {
  display: grid;
  grid-template-columns: 1fr 1fr 1fr;
}```
<pre class="snippet-code-html lang-html prettyprint-override"><code><div>

  <li>industry. Lorem Ipsum has been the industry's standard dummy text ever since the 1500s, when an unknown </li>
  <li>Lorem ever since the 1500s, when an unknown </li>
  <li>Lorem Ipsum is simply dummy text of the printing and typesetting industry. Lorem Ipsum has been the industry's standard dummy text ever since the 1500s, when an unknown </li>
  <li>Lorem Ipsum is simply dummy text of the printing and typesetting industry. </li>
  <li> unknown </li>
  <li>Lorem Ipsum is simply dummy text of thver since the 1500s, when an unknown </li>
  <li>L printing and typesetting industry. Lorem Ipsum has been the industry's standard dummy text ever since the 1500s, when an unknown </li>
  <li>Lorem Ipsum is simply dummy text of the printing and typesettinry's standard dummy text ever since the 1500s, when an unknown </li>
  <li>Lorem Ipsum is simply dummy text of Ipsum has been the industry's standard dummy text ever since the 1500s, when an unknown </li>
  <li> when an unknown </li>
  <li>Lorem </li>

  </ul>

</div></code>```

答:
I can't change it, I have this ul li structure and I have to sort it through the grid

 text

 text

 text

 text

 text

 text

Body:
https://codepen.io/daniel-brown/pen/QWjXpXw

I need this li, to have their own height and not someone else’s

Expected outcome

div {
  display: grid;
  grid-template-columns: 1fr 1fr 1fr;
}

industry. Lorem Ipsum has been the industry's standard dummy text ever since the 1500s, when an unknown

Lorem ever since the 1500s, when an unknown

Lorem Ipsum is simply dummy text of the printing and typesetting industry. Lorem Ipsum has been the industry's standard dummy text ever since the 1500s, when an unknown

Lorem Ipsum is simply dummy text of the printing and typesetting industry.

 unknown

Lorem Ipsum is simply dummy text of thver since the 1500s, when an unknown

L printing and typesetting industry. Lorem Ipsum has been the industry's standard dummy text ever since the 1500s, when an unknown

Lorem Ipsum is simply dummy text of the printing and typesettinry's standard dummy text ever since the 1500s, when an unknown

Lorem Ipsum is simply dummy text of Ipsum has been the industry's standard dummy text ever since the 1500s, when an unknown

 when an unknown

Lorem


问:
Checking for duplicate data in SQL Server
Please don't ask me why but there is a lot of duplicate data where every field is duplicated.

For example

```<code>alex, 1
alex, 1
liza, 32
hary, 34
</code>```

I will need to eliminate from this table one of the alex, 1 rows

I know this algorithm will be very ineffecient, but it does not matter. I will need to remove duplicate data.

What is the best way to do this? Please keep in mind I do not have 2 fields, I actually have about 10 fields to check on.

答:
Method A.  You can get a deduped version of your data using

```<code>SELECT field1, field2, ...
INTO Deduped
FROM Source
GROUP BY field1, field2, ...
</code>```

for example, for your sample data,

```<code>SELECT name, number
FROM Source
GROUP BY name, number
</code>```

yields

```<code>alex    1
hary    34
liza    32
</code>```

then simply delete the old table, and rename the new one.  Of course, there are a number of fancy in-place solutions, but this is the clearest way to do it.

Method B. An in-place method is to create a primary key and delete duplicates that way. For example, you can

```<code>ALTER TABLE Source ADD sid INT IDENTITY(1,1);
</code>```

which makes Source look like this

```<code>alex    1   1
alex    1   2
liza    32  3
hary    34  4
</code>```

then you can use

```<code>DELETE FROM Source
WHERE  sid NOT IN
  (SELECT MIN(sid)
   FROM  Source
   GROUP BY name, number)
</code>```

which will give the desired result.  Of course, "NOT IN" is not exactly the most efficient, but it will do the job.  Alternatively, you can LEFT JOIN the grouped table (maybe stored in a TEMP table), and do the DELETE that way.

Body:
Please don't ask me why but there is a lot of duplicate data where every field is duplicated.

For example

alex, 1
alex, 1
liza, 32
hary, 34

I will need to eliminate from this table one of the alex, 1 rows

I know this algorithm will be very ineffecient, but it does not matter. I will need to remove duplicate data.

What is the best way to do this? Please keep in mind I do not have 2 fields, I actually have about 10 fields to check on.

Body:
Method A.  You can get a deduped version of your data using

SELECT field1, field2, ...
INTO Deduped
FROM Source
GROUP BY field1, field2, ...

for example, for your sample data,

SELECT name, number
FROM Source
GROUP BY name, number

yields

alex    1
hary    34
liza    32

then simply delete the old table, and rename the new one.  Of course, there are a number of fancy in-place solutions, but this is the clearest way to do it.

Method B. An in-place method is to create a primary key and delete duplicates that way. For example, you can

ALTER TABLE Source ADD sid INT IDENTITY(1,1);

which makes Source look like this

alex    1   1
alex    1   2
liza    32  3
hary    34  4

then you can use

DELETE FROM Source
WHERE  sid NOT IN
  (SELECT MIN(sid)
   FROM  Source
   GROUP BY name, number)

which will give the desired result.  Of course, "NOT IN" is not exactly the most efficient, but it will do the job.  Alternatively, you can LEFT JOIN the grouped table (maybe stored in a TEMP table), and do the DELETE that way.


问:
A column-vector y was passed when a 1d array was expected error while doing SVM?
I am creating a SVM model with one independent variable X and dependent variable y.I performed feature scaling as both the data variable was not on same scale.

Now when i am training a model on dataset i am getting error as :

DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
  y = column_or_1d(y, warn=True).

Below are the code snippet from which i am getting error:

```<code># Building Model on whole dataset

from sklearn.svm import SVR
regressor = SVR(kernel='rbf')
regressor.fit(X,y)
</code>```

Entire file:

```<code># Importing Libraries

import numpy as np
import matplotlib.pyplot as plt
import pandas as pd

# import data Set

dataset = pd.read_csv('Position_Salaries.csv')
X = dataset.iloc[:, 1:-1].values
y = dataset.iloc[:, -1].values

# Feature Scaling

y = y.reshape(len(y), 1)

# Feature Scaling

from sklearn.preprocessing import StandardScaler
sc_X = StandardScaler()
sc_y = StandardScaler()
X = sc_X.fit_transform(X)
y = sc_y.fit_transform(y)

# Building Model on training dataset

from sklearn.svm import SVR
regressor = SVR(kernel='rbf')
regressor.fit(X,y)
</code>```

File on which i am training my model

答:
You have one feature (variable) so indeed the line y = y.reshape(-1, 1) is needed.

```<code># Importing Libraries

import numpy as np
import matplotlib.pyplot as plt
import pandas as pd

# import data Set

dataset = pd.read_csv('Position_Salaries.csv')
X = dataset.iloc[:, 1:-1].values
y = dataset.iloc[:, -1].values

y = y.reshape(-1, 1)

# Feature Scaling

from sklearn.preprocessing import StandardScaler
sc_X = StandardScaler()
sc_y = StandardScaler()
X = sc_X.fit_transform(X)
y = sc_y.fit_transform(y)

# Building Model on training dataset

from sklearn.svm import SVR
regressor = SVR(kernel='rbf')
regressor.fit(X,y)
</code>```

Verify that the model was fitted:

```<code>regressor.get_params()

{'C': 1.0,
 'cache_size': 200,
 'coef0': 0.0,
 'degree': 3,
 'epsilon': 0.1,
 'gamma': 'scale',
 'kernel': 'rbf',
 'max_iter': -1,
 'shrinking': True,
 'tol': 0.001,
 'verbose': False}
</code>```

The warning that you get is:

/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/sklearn/utils/validation.py:73:
  DataConversionWarning: A column-vector y was passed when a 1d array
  was expected. Please change the shape of y to (n_samples, ), for
  example using ravel().   return f(**kwargs)

and it is a DataConversionWarning that means that the algorithm did not converge successfully.

Body:
I am creating a SVM model with one independent variable X and dependent variable y.I performed feature scaling as both the data variable was not on same scale.

Now when i am training a model on dataset i am getting error as :

DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
  y = column_or_1d(y, warn=True).

Below are the code snippet from which i am getting error:

# Building Model on whole dataset

from sklearn.svm import SVR
regressor = SVR(kernel='rbf')
regressor.fit(X,y)

Entire file:

# Importing Libraries

import numpy as np
import matplotlib.pyplot as plt
import pandas as pd

# import data Set

dataset = pd.read_csv('Position_Salaries.csv')
X = dataset.iloc[:, 1:-1].values
y = dataset.iloc[:, -1].values

# Feature Scaling

y = y.reshape(len(y), 1)

# Feature Scaling

from sklearn.preprocessing import StandardScaler
sc_X = StandardScaler()
sc_y = StandardScaler()
X = sc_X.fit_transform(X)
y = sc_y.fit_transform(y)

# Building Model on training dataset

from sklearn.svm import SVR
regressor = SVR(kernel='rbf')
regressor.fit(X,y)

File on which i am training my model

Body:
You have one feature (variable) so indeed the line y = y.reshape(-1, 1) is needed.

# Importing Libraries

import numpy as np
import matplotlib.pyplot as plt
import pandas as pd

# import data Set

dataset = pd.read_csv('Position_Salaries.csv')
X = dataset.iloc[:, 1:-1].values
y = dataset.iloc[:, -1].values

y = y.reshape(-1, 1)

# Feature Scaling

from sklearn.preprocessing import StandardScaler
sc_X = StandardScaler()
sc_y = StandardScaler()
X = sc_X.fit_transform(X)
y = sc_y.fit_transform(y)

# Building Model on training dataset

from sklearn.svm import SVR
regressor = SVR(kernel='rbf')
regressor.fit(X,y)

Verify that the model was fitted:

regressor.get_params()

{'C': 1.0,
 'cache_size': 200,
 'coef0': 0.0,
 'degree': 3,
 'epsilon': 0.1,
 'gamma': 'scale',
 'kernel': 'rbf',
 'max_iter': -1,
 'shrinking': True,
 'tol': 0.001,
 'verbose': False}

The warning that you get is:

/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/sklearn/utils/validation.py:73:
  DataConversionWarning: A column-vector y was passed when a 1d array
  was expected. Please change the shape of y to (n_samples, ), for
  example using ravel().   return f(**kwargs)

and it is a DataConversionWarning that means that the algorithm did not converge successfully.