i want extract processed list of dictionary values (list<dictionary<string,object>>) raw list dictionary values(list<dictionary<string,object>>) .
the raw dict may contain string/numeric value
eg:
dictionary<string, object> rawlistdict = new dictionary<string, object>(); rawlistdict.add("product","apple"); rawlistdict.add("region", "west"); rawlistdict.add("profit", 90); raw list:
apple west 90
apple east 10
apple west 80
processed list :
apple west 170
apple east 10
consider list dictionaries having same product , region, want single dict adding "profit" when "product" & "region" same. (i.e) list of dictionaries similar items grouped single dictionary without duplicates
note : raw list can grow beyond 30k entries. :-(
i have implemented logic brute-force technique , consuming lot of memory , time .is there way in linq style or other approach reduce time , memory ?
edit : preferring dictionary since number of members/keys known @ run time.
code have implemented :
//get fields used combining values var nonmeasurablefields = report.datagrid_fields. where(field => field.datatype.equals(imfielddatatype.string_value) || field.datatype.equals(imfielddatatype.date_value)). select(field => field.name).tolist(); if (nonmeasurablefields != null && nonmeasurablefields.count > 0) { #region outer loop (int index = 0; index < processeddata.count; index++) { var basedict = processeddata.elementat(index); dictionary<string, object> comparedict = null; #region recursive loop (int recursiveindex = index + 1; recursiveindex < processeddata.count; recursiveindex++) { comparedict = processeddata.elementat(recursiveindex); int matchescount = 0; #region comparison logic foreach (var key in nonmeasurablefields) { var basedictvalue = basedict[key]; var comparedictvalue = comparedict[key]; if (basedictvalue == null && comparedictvalue == null) { matchescount++; } else { if (basedictvalue != null && comparedictvalue == null) { matchescount = 0; } else if (basedictvalue == null && comparedictvalue != null) { matchescount = 0; } else if (basedictvalue != null && comparedictvalue != null) { if (basedictvalue.equals(comparedictvalue)) { matchescount++; } else { matchescount = 0; } } } } #endregion #region if match -- combine if (matchescount == nonmeasurablefields.count) { #region combine logic // combine 2 dictionary .. processeddata.remove(basedict); processeddata.remove(comparedict); // combine base , compare dict dictionary<string, object> combineddict = new dictionary<string, object>(); var keyneededindict = basedict.keys.tolist(); foreach (var key in keyneededindict.tolist()) { if (nonmeasurablefields.contains(key)) { combineddict.add(key, basedict[key]); } else { object value = convert.todouble(basedict[key]) + convert.todouble(comparedict[key]); combineddict.add(key, value); } } processeddata.add(combineddict); index = -1; // resetting looping index merging works values recursiveindex = -1; // ensuring values considered @ least once whenever // change made list (i.e merging dict) break; #endregion } else { // no matches // continue next } #endregion } #endregion } #endregion } note : i have information of key(value key) of string type , numeric type. sample provided demonstration purpose only.the keys , values known @ run time. supposed combine 2 dictionaries if string values equal.i add numeric values while combining it.
edit 2 : dictionaries in list have same keys no values discarded.dictionaries same values combined.
so, have an
ienumerable<idictionary<string, object>> and want merge dictionaries, based on key set.
you need keys of dictionary form keyset, can group dictionaries appropriately.
you need need delegate function aggragate each of non key set values.
on basis, need function this, work,
ienumerable<idictionary<string, object>> merger( ienumerable<idictionary<string, object>> source, ienumerable<string> keys, idictionary<string, func<ienumerable<object>, object>> aggregators) { var grouped = source.groupby(d => string.join("|", keys.select(k => d[k]))); foreach(var g in grouped) { var result = new dictionary<string, object>(); var first = g.first(); foreach(var key in keys) { result.add(key, first[key]); } foreach(var in aggregators) { result.add(a.key, a.value(g.select(i => i[a.key]))); } yield return result; } } so, if working example data, you'd call this
var processeddictionaries = merger( rawlistdict, new[] { "product", "region" }, new dictionary<string, func<ienumerable<object>, object>> { { "profit", objects => objects.cast<int>().sum() } }); if values string representations of doubles, might prepare aggregators this,
var aggregators = new dictionary<string, func<ienumerable<object>, object>>(); aggregators.add( "profit", objects => objects.cast<string>().sum(s => double.parse(s))); 
Comments
Post a Comment