Returning DataTables in WCF/.NET




I have a WCF service from which I want to return a DataTable. I know that this is often a highly-debated topic, as far as whether or not returning DataTables is a good practice. Let's put that aside for a moment.


When I create a DataTable from scratch, as below, there are no problems whatsoever. The table is created, populated, and returned to the client, and all is well:



```
[DataContract]
public DataTable GetTbl()
{
    DataTable tbl = new DataTable("testTbl");
    for(int i=0;i<100;i++)
    {
        tbl.Columns.Add(i);
        tbl.Rows.Add(new string[]{"testValue"});
    }
    return tbl;
}

```

However, as soon as I go out and hit the database to create the table, as below, I get a CommunicationException "The underlying connection was closed: The connection was closed unexpectedly."



```
[DataContract]
public DataTable GetTbl()
{
    DataTable tbl = new DataTable("testTbl");
    //Populate table with SQL query

    return tbl;
}

```

The table is being populated correctly on the server side. It is significantly smaller than the test table that I looped through and returned, and the query is small and fast - there is no issue here with timeouts or large data transfer. The same exact functions and DataContracts/ServiceContracts/BehaviorContracts are being used.


Why would the way that the table is being populated have any bearing on the table returning successfully?
For anyone having similar problems, I have solved my issue. It was several-fold.


* As Darren suggested and Paul backed up, the Max..Size properties in the configuration needed to be enlarged. The SvcTraceViewer utility helped in determining this, but it still does not always give the most helpful error messages.
* It also appears that when the Service Reference is updated on the client side, the configuration will sometimes not update properly (e.g. Changing config values on the server will not always properly update on the client. I had to go in and change the Max..Size properties multiple times on both the client and server sides in the course of my debugging)
* For a DataTable to be serializable, it needs to be given a name. The default constructor does not give the table a name, so:



```
return new DataTable();

```

will not be serializable, while:



```
return new DataTable("someName");

```

will name the table whatever is passed as the parameter. 


Note that a table can be given a name at any time by assigning a string to the `TableName` property of the DataTable.



```
var table = new DataTable();
table.TableName = "someName";

```


Hopefully that will help someone.
If I am not wrong, you are trying to read or loop through the rows from datatable. This error states that the datasource(DataTable) is not having any rows. To avoid this exception use this before you read or loop through datatable:  


```
if(dt.Rows.Count  > 0 ){
     //Do your stuff here.
}
```
  

  

  

--Amit A couple of notes to get started:



> 
> .apply(literal\_eval)
> 
> 
> 


Wouldn't this be better as `map`?



> 
> I then re-create the Dask DF:
> 
> 
> ddf = dd.from\_pandas(ddf\_out, npartitions=nCores)
> 
> 
> 


`ddf_out` was already a dask dataframe, I don't know why you had to do this.



> 
> The columns in result seem to match those in metadir.
> 
> 
> 


The value of `result.columns` is taken from the meta you provided, no computing happens until you ask for it (dask is lazy in most operations). Does the ValueError exception not provide further information?


Here is a full example



```
x = ({'F9_07_PZ_COMP_DIRECT': '0',
  'F9_07_PZ_DIRTRSTKEY_NAME': 'DEBRA MEALY',
  'F9_07_PZ_COMP_OTHER': '0',
  'F9_07_PZ_COMP_RELATED': '0',
  'F9_07_PZ_TITLE': 'CHAIR PERSON',
  'F9_07_PZ_AVE_HOURS_WEEK': '1.00',
  'F9_07_PC_TRUSTEE_INDIVIDUAL': 'X'},
 {'F9_07_PZ_COMP_DIRECT': '0',
  'F9_07_PZ_DIRTRSTKEY_NAME': 'HELEN GORDON',
  'F9_07_PZ_COMP_OTHER': '0',
  'F9_07_PZ_COMP_RELATED': '0',
  'F9_07_PZ_TITLE': 'VICE CHAIR',
  'F9_07_PZ_AVE_HOURS_WEEK': '1.00',
  'F9_07_PC_TRUSTEE_INDIVIDUAL': 'X'})
df = pd.DataFrame({'a': x})
d = dd.from_pandas(df, 1)
meta = pd.DataFrame(columns=['F9_07_PZ_COMP_DIRECT', 
       'F9_07_PZ_DIRTRSTKEY_NAME',
       'F9_07_PZ_COMP_OTHER', 'F9_07_PZ_COMP_RELATED', 'F9_07_PZ_TITLE',
       'F9_07_PZ_AVE_HOURS_WEEK', 'F9_07_PC_TRUSTEE_INDIVIDUAL'], dtype="O")
d.map_partitions(lambda df: df.a.apply(pd.Series), meta=meta).compute()

```

How did I know what `meta` to use? I applied the function to the pandas dataframe - you might use a small piece of the dataframe to do this.


Some additional notes:


* it is an anti-pattern to load the data with pandas, pass to dask workers and then gather the whole result back to a pandas (in memory) dataframe, you are very unlikely to see a speedup this way and may incur a lot of overhead. You are better off loading with something like `dd.read_csv`, and aggregating or writing with dask functions too. Only `compute()` on something that will be small or not return anything (because it involves writing the output). The official examples do not use from\_pandas.
* string and dict processing are python methods, and so hold the interpreter lock (GIL) of any python function: threads will not actually run in parallel. To get parallelism, you need to run in processes, which is most easily achieved using <https://docs.dask.org/en/latest/setup/single-distributed.html>
* the distributed scheduler also gives you access to the dashboard, which has a lot of useful information to diagnose how your system is running. You can also configure a lot about its behaviour, in case you have firewall rules you need to obey. Hi  

This is simple. First check whether you are able to connect Sql Server through SSMS or not.If you are able to connect through SSMS, then there may be a problem in connection string. If you are not able to connect through SSMS, then you had some problem with SQL Environment. I just posted an example of this over in a similar thread about JTable [here](/software-development/java/threads/373078/1605358#post1605358). Perhaps it will help. strangely enough it did in a strange way. i reinstalled, set up everything again from scratch (agents too) and got the same ‘not connected’ error.

i got frustrated and left it overnight.

the following morning in the office i checked and it discovered both services!

so i guess you could close this… unfortunately that's not how it works. it's db mapping, and we're talking about joined-tables hierarchy. so the "other side" of the association must be in the referenced table (the detail one, not custom detail). i suspect that the resource\_exhausted message may have originated from a back-pressuring committer node ([code](https://github.com/digital-asset/daml/blob/926fb5997a795e7f80443a9a3ba7898b1568d158/ledger/participant-integration-api/src/main/scala/platform/apiserver/services/apisubmissionservice.scala#l155-l157)). if that’s the case, i believe the approach should be the same: change the configuration if you need and make sure to handle those errors with the appropriate retry strategy. yeah that happened time ago, with a girl from europe, we "met" online we had the "greatest" connection, it felt like we knew each other from years before, but there was something we didn't talk and is "well this is great but you are in the other part of the world now," of course money , distance, even the fact we were completely strangers, we didn't talk never, (for months) where things were going, long story short, she got into another more time consuming job, things begin to get cold quickly, then one day she just didn't reply anymore, 2nd experience, i met her online, she visited me couple of times while been here , yeah she was from europe too, (for distance reference purpose ) we had great time , but things were like this is great we just have a very different life each one on their own country, we are incredible far to make things work at least for now, we still keep in contact for years as very good friends, good luck probably you both make it and find the way things work "faster" : ) hi paul,

i believe the financial reporting covers in quite a lot of detail exactly what you are looking for.

from what i can tell you aren’t using templates in your tables…

[[image]](https://forum.enterprisedna.co/uploads/db7134/original/2x/5/559a333f85c159c8d9bec9235ca5c45997d08858.png "image")

if you want to get a certain order and custom table then you really need to use the template approach covered in t… Lets say you want to send a table with a single column of GUIDs.


First we need to create a structure using [SqlMetaData](https://msdn.microsoft.com/en-us/library/microsoft.sqlserver.server.sqlmetadata(v=vs.110).aspx) which represents the schema of the table (columns).



```
var tableSchema = new List<SqlMetaData>(1)
{
  new SqlMetaData("Id", SqlDbType.UniqueIdentifier)
}.ToArray();

```

Next you create a list of records that match the schema using [SqlDataRecord](https://msdn.microsoft.com/en-us/library/microsoft.sqlserver.server.sqldatarecord(v=vs.110).aspx).



```
var tableRow = new SqlDataRecord(tableSchema);
tableRow.SetGuid(0, Guid.NewGuid());
var table = new List<SqlDataRecord>(1)
{
  tableRow
};

```

Then create the [SqlParameter](https://msdn.microsoft.com/en-us/library/system.data.sqlclient.sqlparameter(v=vs.110).aspx):



```
var parameter = new SqlParameter();
parameter.SqlDbType = SqlDbType.Structured;
parameter.ParameterName = "@UserIds";
parameter.Value = table;

var parameters = new SqlParameter[1]
{
  parameter
};

```

Then simply call the stored procedure by using the [Database.SqlQuery](https://msdn.microsoft.com/en-us/library/gg696545(v=vs.113).aspx).



```
IEnumerable<ReturnType> result;
using (var myContext = new DbContext())
{
  result = myContext.Database.SqlQuery<User>("GetUsers", parameters)
    .ToList();         // calls the stored procedure
    // ToListAsync();  // Async
{

```

In SQL Server, create your User-Defined Table Type (I suffix them with TTV, Table Typed Value):



```
CREATE TYPE [dbo].[UniqueidentifiersTTV] AS TABLE(
  [Id] [uniqueidentifier] NOT NULL
)
GO

```

Then specify the type as a parameter (don't forget, Table Type Values have to be readonly!):



```
CREATE PROCEDURE [dbo].[GetUsers] (
  @UserIds [UniqueidentifiersTTV] READONLY
) AS
BEGIN
  SET NOCOUNT ON

  SELECT u.* -- Just an example :P
  FROM [dbo].[Users] u
  INNER JOIN @UserIds ids On u.Id = ids.Id
END

```