text:
Fast Apps, Microsoft StylePosted: April 28, 2014 Filed under: Microsoft, Musings, Network Programming, News, System Programming Leave a comment
That's what I exclaimed at least a couple of times this morning as I sat at a table in a makeshift "team room" in building 43 at Microsoft's Redmond campus. What was the exclamation for? Well, over the past 3 months, I've been working on a quick strike project with a new team, and today we finally announced our "Public Preview". Or, if you want to get right to the product: Cloud App Discovery
I'm not a PM or marketing type, so it's best to go and read the announcement for yourself if you want to get the official spiel on the project. Here, I want to write a bit about the experience of coming up with a project, in short order, in the new Microsoft.
It all started back in January for me. I was just coming off another project, and casting about for the next hardest 'mission impossible' to jump on. I had a brief conversation with a dev manager who posed the question; "Is it possible to reestablish the 'perimeter' for IT guys in this world of cloud computing"? An intriguing question. The basic problem was, if you go to a lot of IT guys, they can barely tell you how many of the people within their corporation are using SalesForce.com, let alone DropBox from a cafe in Singapore. Forget the notion of even trying to control such access. The corporate 'firewall' is almost nothing more than a quartz space heater at this point, preventing very little, and knowing about even less.
So, with that question in mind, we laid out 3 phases of development. Actually, they were already laid out before I joined the party (by a couple of weeks), so I just heard the pitch. It was simple, the first phase of development is to see if we can capture network traffic, using various means, and project it up to the cloud where we could use some machine learning to give an admin a view of what's going on.
Conveniently sidestepping any objections actual employees might have with this notion, I got to thinking on how it could be done.
For my part, we wanted to have something sitting on the client machine (a windows machine that the user is using), which will inspect all network traffic coming and going, and generate some reports to be sent up to the cloud. Keep in mind, this is all consented activity, the employee gets to opt in to being monitored in this way. All in the open and up front.
At the lowest level, my first inclination was to use a raw socket to create a packet sniffer, but Windows has a much better solution these days, built for exactly this purpose. The Windows Filter Platform, allows you to create a 'filter' which you can configure to callout to a function whenever there is traffic. My close teammate implemented that piece, and suddenly we had a handle on network packets.
We fairly quickly decided on an interface between that low level packet sniffing, and the higher level processor. It's as easy as this:
int WriteBytes(char *buff, int bufflen); int ReadBytes(char *buff, int bufflen, int &bytesRead);
I'm paraphrasing a bit, but it really is that simple. What's it do? Well, the fairly raw network packets are sent into 'WriteBytes', some processing is done, and a 'report' becomes available through 'ReadBytes'. The reports are a JSON formatted string which then gets turned into the appropriate thing to be sent up to the cloud.
The time it took from hearing about the basic product idea, to a prototype of this thing was about 3 weeks.
What do I do once I get network packets? Well, the network packets represent a multiplexed stream of packets, as if I were a NIC. All incoming, outgoing, all TCP ports. Once I receive some bytes, I have to turn it back into individual streams, then start doing some 'parsing'. Right now we handle http and TLS. For http, I do full http parsing, separating out headers, reading bodies, and the like. I did that by leveraging the http parsing work I had done for TINN already. I used C++ in this case, but it's all relatively the same.
TLS is a different story. At this 'discovery' phase, it was more about simple parsing. So, reading the record layer, decoding client_hello and server_hello, certificate, and the like. This gave me a chance to implement TLS processing using C++ instead of Lua. One of the core components that I leveraged was the byte order aware streams that I had developed for TINN. That really is the crux of most network protocol handling. If you can make herds or tails of what the various RFCs are saying, it usually comes down to doing some simple serialization, but getting the byte ordering is the hardest part. 24-bit big endian integers?
At any rate, http parsing, fairly quick. TLS client_hello, fast enough, although properly handling the extensions took a bit of time. At this point, we'd be a couple months in, and our first partners get to start kicking the tires.
For such a project, it's very critical that real world customers are involved really early, almost sitting in our design meetings. They course corrected us, and told us what was truly important and annoying about what we were doing, right from day one.
From the feedback, it becomes clear that getting more information, like the amount of traffic flowing through the pipes is as interesting as the meta information, so getting the full support for flows becomes a higher priority. For the regular http traffic, no problem. The TLS becomes a bit more interesting. In order to deal with that correctly, it becomes necessary to suck in more of the TLS implementation. Read the server_hello, and the certificate information. Well, if you're going to read in the cert, you might as well get the subject common name out so you can use that bit of meta information. Now comes ASN.1 (DER) parsing, and x509 parsing. That code took about 2 weeks, working "nights and weekends" while the other stuff was going on. It took a good couple of weeks not to integrate, but to write enough test cases, with real live data, to ensure that it was actually working correctly.
The last month was largely a lot of testing, making sure corner cases were dealt with and the like. As the client code is actually deployed to a bunch of machines, it really needed to be rock solid, no memory leaks, no excessive resource utilization, no CPU spiking, just unobtrusive, quietly getting the job done.
So, that's what it does.
Now, I've shipped at Microsoft for numerous years. The fastest cycles I've usually dealt with are on the order of 3 months. That's usually for a product that's fairly mature, has plenty of engineering system support, and a well laid out roadmap. Really you're just turning the crank on an already laid out plan.
This AppDiscovery project has been a bit different. It did not start out with a plan that had a 6 month planning cycle in front of it. It was a hunch that we could deliver customer value by implementing something that was challenging enough, but achievable, in a short amount of time.
So, how is this different than Microsoft of yore? Well, yes, we've always been 'customer focused', but this is to the extreme. I've never had customers this involved in what I was doing this early in the development cycle. I mean literally, before the first prototypical bits are even dry, the PM team is pounding on the door asking "when can I give it to the customers?". That's a great feeling actually.
The second thing is how much process we allowed ourselves to use. Recognizing that it's a first run, and recognizing that customers might actually say "mehh, not interested", it doesn't make sense to spin up the classic development cycle which is meant to maintain a product for 10-14 years. A much more streamlined lifecycle which favors delivering quality code and getting customer feedback, is what we employed. If it turns out that customers really like the product, then there's room to fit the cycle to a cycle that is more appropriate for longer term support.
The last thing that's special is the amount of leveraging Open Source we are allowing ourselves these days. Microsoft has gone full tilt on OpenSource support. I didn't personally end up using much myself, but we are free to use it elsewhere (with some legal guidelines). This is encouraging, because for crypto, I'm looking forward to using things like SipHash, and ChaCha20, which don't come natively with the Microsoft platform.
Overall, as Microsoft continues to evolve and deliver 'customer centric' stuff, I'm pretty excited and encouraged that we'll be able to use this same model time and again to great effect. Microsoft has a lot of smart engineers. Combined with some new directives about meeting customer expectations at the market, we will surely be cranking out some more interesting stuff.
I've implemented some interesting stuff while working on this project, some if it I'll share here.
Device Iteration with Functional ProgrammingPosted: January 29, 2014 Filed under: Core Programming, Lua, LuaJIT, Microsoft, System Programming, TINN | Tags: device enumeration, functional programming, lua fun, tinn, windows Leave a comment
One of the great pleasures I have in life is learning something new. There's nothing greater than those 'light bulb goes on' moments as you realize something and gain a much deeper understanding than you had before.
Well, a little while ago, there was an announcement of this thing called Lua Fun. Lua Fun is a large set of functions which make functional programming in Lua really easy. It has the usual suspects such as map, reduce, filter, each, etc. If you read the documentation, you get a really good understanding of how iterators work in Lua, and more importantly, how LuaJIT is able to fold and manipulate things in hot loops such that the produced code is much tighter than anything I could possibly write in C, or any other language I so happen to use.
So, now I'm a fan of Lua Fun, and I would encourage anyone who's both into Lua, and functional programming to take a look.
How to use it? I've been enumerating various types of things in the Windows system of late. Using the MMDevice subsystem, I was able to get an enumeration of the audio devices (that took a lot of COM work). What about displays, and disk drives, and USB devices, and… Yes, each one of those things has an attendant API which will facilitate monitoring said category. But, is there one to rule them all? Well yes, as it turns out, in most cases what these various APIs are doing is getting information out of the System Registry, and just presenting it reasonably.
There is a way to enumerate all the devices in the system. You know, like when you bring up the Device Manager in Windows, and you see a tree of devices, and their various details. The stage is set, how do you do that? I created a simple object that does the grunt work of enumerating the devices in the system.
The DeviceRecordSet is essentially a query. Creating an instance of the object just gives you a handle onto making query requests. Here is the code:
local ffi = require("ffi") local bit = require("bit") local bor = bit.bor; local band = bit.band; local errorhandling = require("core_errorhandling_l1_1_1"); local SetupApi = require("SetupApi") local WinNT = require("WinNT") local DeviceRecordSet = {} setmetatable(DeviceRecordSet, { __call = function(self, ...) return self:create(...) end, }) local DeviceRecordSet_mt = { __index = DeviceRecordSet, } function DeviceRecordSet.init(self, rawhandle) print("init: ", rawhandle) local obj = { Handle = rawhandle, } setmetatable(obj, DeviceRecordSet_mt) return obj; end function DeviceRecordSet.create(self, Flags) Flags = Flags or bor(ffi.C.DIGCF_PRESENT, ffi.C.DIGCF_ALLCLASSES) local rawhandle = SetupApi.SetupDiGetClassDevs( nil, nil, nil, Flags); if rawhandle == nil then return nil, errorhandling.GetLastError(); end return self:init(rawhandle) end function DeviceRecordSet.getNativeHandle(self) return self.Handle; end function DeviceRecordSet.getRegistryValue(self, key, idx) idx = idx or 0; did = ffi.new("SP_DEVINFO_DATA") did.cbSize = ffi.sizeof("SP_DEVINFO_DATA"); --print("HANDLE: ", self.Handle) local res = SetupApi.SetupDiEnumDeviceInfo(self.Handle,idx,did) if res == 0 then local err = errorhandling.GetLastError() --print("after SetupDiEnumDeviceInfo, ERROR: ", err) return nil, err; end local regDataType = ffi.new("DWORD[1]") local pbuffersize = ffi.new("DWORD[1]",260); local buffer = ffi.new("char[260]") local res = SetupApi.SetupDiGetDeviceRegistryProperty( self:getNativeHandle(), did, key, regDataType, buffer, pbuffersize[0], pbuffersize); if res == 0 then local err = errorhandling.GetLastError(); --print("after GetDeviceRegistryProperty, ERROR: ", err) return nil, err; end --print("TYPE: ", regDataType[0]) if (regDataType[0] == 1) or (regDataType[0] == 7) then return ffi.string(buffer, pbuffersize[0]-1) elseif regDataType[0] == ffi.C.REG_DWORD_LITTLE_ENDIAN then return ffi.cast("DWORD *", buffer)[0] end return nil; end function DeviceRecordSet.devices(self, fields) fields = fields or { {ffi.C.SPDRP_DEVICEDESC, "description"}, {ffi.C.SPDRP_MFG, "manufacturer"}, {ffi.C.SPDRP_DEVTYPE, "devicetype"}, {ffi.C.SPDRP_CLASS, "class"}, {ffi.C.SPDRP_ENUMERATOR_NAME, "enumerator"}, {ffi.C.SPDRP_FRIENDLYNAME, "friendlyname"}, {ffi.C.SPDRP_LOCATION_INFORMATION , "locationinfo"}, {ffi.C.SPDRP_LOCATION_PATHS, "locationpaths"}, {ffi.C.SPDRP_PHYSICAL_DEVICE_OBJECT_NAME, "objectname"}, {ffi.C.SPDRP_SERVICE, "service"}, } local function closure(fields, idx) local res = {} local count = 0; for _it, field in ipairs(fields) do local value, err = self:getRegistryValue(field[1], idx) if value then count = count + 1; res[field[2]] = value; end end if count == 0 then return nil; end return idx+1, res; end return closure, fields, 0 end return DeviceRecordSet
The 'getRegistryValue()' function is the real workhorse of this object. That's what gets your values out of the system registry. The other function of importance is 'devices()'. This is an iterator.
There are a couple of things of note about this iterator. First of all, it does not require 'up values' to be held onto. All that means is that everything the iterator needs to operate is carried in the return values from the function. The 'state' if you will, is handed in fresh every time the 'closure()' is called. This is the key to creating an iterator that will work well with Lua Fun.
By default, this iterator will return quite a few (but not all) fields related to each object, and it will return all the objects. This is ok, because there are typically less than 150 objects in any given system.
Now, I want to do various queries against this set without much fuss. This is where Lua Fun, and functional programming in general, really shines.
First, a little setup:
--test_enumdevices.lua local ffi = require("ffi") local DeviceRecordSet = require("DeviceRecordSet") local serpent = require("serpent") local Functor = require("Functor") local fun = require("fun")() local drs = DeviceRecordSet(); local function printIt(record) print("==========") each(print, record) print("----------") end
This creates an instance of the DeviceRecordSet object, which will be used in the queries. Already the printIt() function is utilizing Lua Fun. The 'each()' function will take whatever it's handed, and perform the function specified. In this case, the 'record' will be a table. So, each will iterate over the table entries and print each one of them out. This is the equivalent of doing:
for k,v in pairs(record)
print(k, v)
I think that simply typing 'each' is a lot simpler and pretty easy to understand.
How about a query then?
-- show everything for every device each(printIt, drs:devices())
In this case, the 'each' is applied to the results of the 'devices()' iterator. For each record coming from the devices iterator, the printIt function will be called, which will in turn print out all the values in the record. That's pretty nice.
What if I don't want to see all the fields in the record, I just want to see the objectname, and description fields. Well, this is a 'map' operation, or a projection in database parlance, so:
-- do a projection on the fields local function projection(x) return {objectname = x.objectname, description = x.description} end each(printIt, map(projection, drs:devices()))
Working from the inside out, for each record coming from the devices() iterator, call the 'projection' function. The return value from the projection function becomes the new record for this iteration. For each of those records, call the printIt function.
Using 'map' is great as you can reshape data in any way you like without much fuss.
Lastly, I want to see only the records that are related to "STORAGE", so…
-- show only certain records local function enumeratorFilter(x) return x.enumerator == "STORAGE" end each(printIt, filter(enumeratorFilter, drs:devices()))
Here, the 'filter' iterator is used. So, again, for each of the records coming from the 'devices()' enumerator, call the 'enumeratorFilter' function. If this function returns 'true' for the record, then it is passed along as the next record for the 'each'. If 'false', then it is skipped, and the next record is tried.
This is pretty powerful, and yet simple stuff. The fact that iterators create new iterators, in tight loops, makes for some very dense and efficient code. If you're interested in why this is so special in LuaJIT, and not many other languages, read up on the Lua Fun documentation.
I've killed two birds with one stone. I have finally gotten to the root of all device iterators. I have also learned how to best write iterators that can be used in a functional programming way. Judicious usage of this mechanism will surely make a lot of my code more compact and readable, as well as highly performant.
Asynchronous DNS lookups on WindowsPosted: January 28, 2014 Filed under: Core Programming, LuaJIT, Microsoft, Network Programming, System Programming, TINN | Tags: async, dns, getaddrinfo, tinn, windows Leave a comment
I began this particular journey because I wanted to do DNS lookups asynchronously on Windows. There is of course a function for that:
The problem I ran into is that unlike the various other Windows functions I've done with async, this one does not use an IO Completion Port. Instead it uses a mechanism called APC (Asynchronouse Procedure Call). With this little bit of magic, you pass in a pointer to a function which will be called in your thread context, kind of in between when other things are happening. Well, given the runtime environment I'm using, I don't think this quite works out. Basically, I'd have a function being called leaving the VM in an unknown state.
So, I got to digging. I figured, how hard can it be to make calls to a DNS server directly? After all, it is nothing more than a network based service with a well known protocol. Once I could make a straight networking call, then I could go back to leveraging IO Completion Ports just like I do for all other IO.
You can view the DNS system as nothing more than a database to which you pose queries. You express your queries using some nice well defined protocol, which is ancient in origin, and fairly effective considering how frequently DNS queries are issued. Although I could man up and write the queries from scratch, Windows helps me here by providing functions that will format the query into a buffer for me.
But, before I get into that, what do the queries look like? What am I looking up? Well, a Domain Name Server serves up translations of names to other names and numbers. For example, I need to find the IP address of http://www.bing.com. I can look for CNAMES (an alias), or 'A' records (direct to an IP address. This gets esoteric and confusing, so a little code can help:
-- Prepare the DNS request local dwBuffSize = ffi.new("DWORD[1]", 2048); local buff = ffi.new("uint8_t[2048]") local wID = clock:GetCurrentTicks() % 65536; local res = windns_ffi.DnsWriteQuestionToBuffer_UTF8( ffi.cast("DNS_MESSAGE_BUFFER*",buff), dwBuffSize, ffi.cast("char *",strToQuery), wType, wID, true )
DnsWriteQuestionToBuffer_UTF8 is the Windows function which helps me to write a DNS query into a buffer, which will then be send to the actual dns server.
wType, represents the type of record you want to be returned. The values might be something like:
wType = ffi.C.DNS_TYPE_A wType = ffi.C.DNS_TYPE_MX - mail records wType = ffi.C.DNS_TYPE_CNAME
There are about a hundred different types that you can query for. The vast majority of the time though, you either looking for 'A', or 'CNAME' records.
The wID is just a unique identifier for the particular query so that if you're issuing several on the same channel, you can check the response to ensure they match up.
OK. Now I have a DNS query stuffed into a buffer, how do I make the query and get the results?
-- Send the request. local IPPORT_DNS = 53; local remoteAddr = sockaddr_in(IPPORT_DNS, AF_INET); remoteAddr.sin_addr.S_addr = ws2_32.inet_addr( "220.127.116.11"); -- create the UDP socket local socket, err = NativeSocket( AF_INET, SOCK_DGRAM, IPPROTO_UDP ); -- send the query local iRes, err = socket:sendTo( ServerAddress, ffi.sizeof(ServerAddress), buff, dwBuffSize[0]);
This little bit of code shows the socket creation, and the actual 'sendTo' call. Of note, the "22.214.171.124" represents the IP address of a well known public DNS server. In this case it is hosted by Level 3, which is a internet services provider. There are of course calls you can make to figure out which DNS server your machine is typically configured to use, but this way the query will always work, no matter which network you are on.
Notice the socket is a UDP socket.
At this point, we're already running cooperatively due to the fact that within TINN, all IO is done cooperatively, without the programmer needing to do much special.
Now to receive the query response back:
-- Try to receive the results local RecvFromAddr = sockaddr_in(); local RecvFromAddrSize = ffi.sizeof(RecvFromAddr); local cbReceived, err = self.Socket:receiveFrom(RecvFromAddr, RecvFromAddrSize, buff, 2048);
Basically just wait for the server to send back a response. Of course, like the sendTo, the receiveFrom works cooperatively, so that if the developer issues several 'spawn' commands, each query could be running in its own task, working cooperatively.
Once you have the response, you can parse out the results. The results come back as a set of records. There are of course functions which will help you to parse these records out. The key here is that the record type is indicated, and its up to the developer to pull out the relevant information.
The complete DNSNameServer class is here:
local ffi = require("ffi") local Application = require("Application") local windns_ffi = require("windns_ffi") local NativeSocket = require("NativeSocket") local ws2_32 = require("ws2_32") local Stopwatch = require("StopWatch") local clock = Stopwatch(); -- DNS UDP port local IPPORT_DNS = 53; local DNSNameServer = {} setmetatable(DNSNameServer, { __call = function(self, ...) return self:create(...) end, }) local DNSNameServer_mt = { __index = DNSNameServer, } function DNSNameServer.init(self, serveraddress) local socket, err = NativeSocket( AF_INET, SOCK_DGRAM, IPPROTO_UDP ); if not socket then return nil, err end local obj = { Socket = socket, ServerAddress = serveraddress, } setmetatable(obj, DNSNameServer_mt) return obj; end function DNSNameServer.create(self, servername) local remoteAddr = sockaddr_in(IPPORT_DNS, AF_INET); remoteAddr.sin_addr.S_addr = ws2_32.inet_addr( servername ); return self:init(remoteAddr) end -- Construct DNS_TYPE_A request, send it to the specified DNS server, wait for the reply. function DNSNameServer.Query(self, strToQuery, wType, msTimeout) wType = wType or ffi.C.DNS_TYPE_A msTimeout = msTimeout or 60 * 1000 -- 1 minute -- Prepare the DNS request local dwBuffSize = ffi.new("DWORD[1]", 2048); local buff = ffi.new("uint8_t[2048]") local wID = clock:GetCurrentTicks() % 65536; local res = windns_ffi.DnsWriteQuestionToBuffer_UTF8( ffi.cast("DNS_MESSAGE_BUFFER*",buff), dwBuffSize, ffi.cast("char *",strToQuery), wType, wID, true ) if res == 0 then return false, "DnsWriteQuestionToBuffer_UTF8 failed." end -- Send the request. local iRes, err = self.Socket:sendTo(self.ServerAddress, ffi.sizeof(self.ServerAddress), buff, dwBuffSize[0]); if (not iRes) then print("Error sending data: ", err) return false, err end -- Try to receive the results local RecvFromAddr = sockaddr_in(); local RecvFromAddrSize = ffi.sizeof(RecvFromAddr); local cbReceived, err = self.Socket:receiveFrom(RecvFromAddr, RecvFromAddrSize, buff, 2048); if not cbReceived then print("Error Receiving Data: ", err) return false, err; end if( 0 == cbReceived ) then return false, "Nothing received" end -- Parse the DNS response received with DNS API local pDnsResponseBuff = ffi.cast("DNS_MESSAGE_BUFFER*", buff); windns_ffi.DNS_BYTE_FLIP_HEADER_COUNTS ( pDnsResponseBuff.MessageHead ); if pDnsResponseBuff.MessageHead.Xid ~= wID then return false, "wrong transaction ID" end local pRecord = ffi.new("DNS_RECORD *[1]",nil); iRes = windns_ffi.DnsExtractRecordsFromMessage_W( pDnsResponseBuff, cbReceived, pRecord ); pRecord = pRecord[0]; local pRecordA = ffi.cast("DNS_RECORD *", pRecord); local function closure() if pRecordA == nil then return nil; end if pRecordA.wType == wType then local retVal = pRecordA pRecordA = pRecordA.pNext return retVal; end -- Find the next record of the specified type repeat pRecordA = pRecordA.pNext; until pRecordA == nil or pRecordA.wType == wType if pRecordA ~= nil then local retVal = pRecordA pRecordA = pRecordA.pNext return retVal; end -- Free the resources if pRecord ~= nil then windns_ffi.DnsRecordListFree( pRecord, ffi.C.DnsFreeRecordList ); end return nil end return closure end function DNSNameServer.A(self, domainToQuery) return self:Query(domainToQuery, ffi.C.DNS_TYPE_A) end function DNSNameServer.MX(self, domainToQuery) return self:Query(domainToQuery, ffi.C.DNS_TYPE_MX) end function DNSNameServer.CNAME(self, domainToQuery) return self:Query(domainToQuery, ffi.C.DNS_TYPE_CNAME) end function DNSNameServer.SRV(self, domainToQuery) return self:Query(domainToQuery, ffi.C.DNS_TYPE_SRV) end return DNSNameServer
Notice at the end there are some convenience functions for a few of the well known DNS record types. The 'Query()' function is generic, and will return records of any type. These convenience functions just make it easier.
And how to use it?
local ffi = require("ffi") local DNSNameServer = require("DNSNameServer") local core_string = require("core_string_l1_1_0") --local serveraddress = "10.211.55.1" -- xfinity local serveraddress = "188.8.131.52" -- level 3 local domains = { "www.nanotechstyles.com", "www.adafruit.com", "adafruit.com", "adamation.com", "www.adamation.com", "microsoft.com", "google.com", "ibm.com", "oracle.com", "sparkfun.com", "apple.com", "netflix.com", "www.netflix.com", "www.us-west-2.netflix.com", "www.us-west-2.prodaa.netflix.com", "news.com", "hardkernel.org", "amazon.com", "walmart.com", "target.com", "godaddy.com", "luajit.org", } local function queryA() local function queryDomain(name) local dns = DNSNameServer(serveraddress) -- ms corporate print("==== DNS A ====> ", name) for record in dns:A(name) do local a = IN_ADDR(); a.S_addr = record.Data.A.IpAddress print(string.format("name: %s\tIP: %s, TTL %d", name, a, record.dwTtl)); end end for _, name in ipairs(domains) do spawn(queryDomain, name) --queryDomain(name) end end local function queryCNAME() local dns = DNSNameServer(serveraddress) -- ms corporate local function queryDomain(name) print("==== DNS CNAME ====> ", name) for record in dns:CNAME(name) do print(core_string.toAnsi(record.pName), core_string.toAnsi(record.Data.CNAME.pNameHost)) end end for _, name in ipairs(domains) do queryDomain(name) end end local function queryMX() local function queryDomain(name) local dns = DNSNameServer(serveraddress) -- ms corporate print("==== DNS MX ====> ", name) for record in dns:MX(name) do print(core_string.toAnsi(record.pName), core_string.toAnsi(record.Data["MX"].pNameExchange)) end end for _, name in ipairs(domains) do spawn(queryDomain, name) end end local function querySRV() local dns = DNSNameServer(serveraddress) -- ms corporate for _, name in ipairs(domains) do print("==== DNS SRV ====> ", name) for record in dns:SRV(name) do print(core_string.toAnsi(record.pName), core_string.toAnsi(record.Data.SRV.pNameTarget)) end end end local function main() queryA(); --queryCNAME(); --queryMX(); --querySRV(); end run(main)
The function queryA() will query for the 'A' records, and print them out. Notice that it has knowledge of the giant union structure that contains the results, and it pulls out the specific information for 'A' records. It will create a new instance of the DNSNameServer for each query. That's not as bad as it might seem. All it amounts to is creating a new UDP socket for each query. Since each query is spawned into its own task, they are all free to run and complete independently, which was the goal of this little exercise.
In the case of the CNAME query, there is only a single socket, and it is used repeatedly, serially, for each query.
The difference between the two styles is noticeable. For the serial case, the queries might 'take a while', because you have to wait for each result to come back before issuing the next query. In the cooperative case, you issue several queries in parallel, so the total time will only be as long as the longest query.
That's a good outcome.
I like this style of programming. You go as low as you can to root out where the system might otherwise block, and you make that part cooperative. That way everything else above it is automatically cooperative. I also like the fact that it feels like I'm getting some parallelism, but I'm not using any of the typical primitives of parallelism, including actual threads, mutexes, and the like.
Well, that's a hefty bit of code, and it serves the purpose I set out, so I'm a happy camper. Now, if I could just turn those unions into tables automatically…
All the pretty little asyncs…Posted: January 24, 2014 Filed under: Core Programming, Lua, LuaJIT, Microsoft, System Programming, TINN | Tags: async, deviceiocontrol, microsoft, tinn, windows Leave a comment
I have gone on about various forms of async for quite some time now. So could there possibly be more? Well, yes of course!
Here's the scenario I want to enable. I want to keep track of my file system activity, sending the various operations to a distant storage facility. I want to do this while a UI displays what's going on, and I want to be able to configure things while its happening, like which events I really care to shadow, and the like.
I don't want to use multiple OS level threads if I can at all avoid them as they will complicate my programming tremendously. So, what to do.
Well, first I'll start with the file tracking business. I have talked about change journals in the past. This is a basic mechanism that Windows has to track changes to the file system. Every single open, close, delete, write, etc, has an entry in this journal. If you're writing a backup program, you'll be using change journals.
The essence of the change journal functionality is usage of the DeviceIoControl() function. Most of us are very familiar with the likes of CreateFile(), ReadFile(), WriteFile(), CloseHandle(), when it comes to dealing with files. But, for everything else, there is this DeviceIOControl() function.
What is a device? Well, you'd be surprised to learn that most things in the Windows OS are represented by 'devices' just like they are in UNIX systems. For example, 'C:', is a device. But, also "DISPLAY1″ is also a device, as are "LCD" and "PhysicalDisk0″. When it comes to controlling devices, the Win32 level API calls will ultimately make DeviceIoControl() calls with various parameters. That's great to know as it allows you to create whatever API you want, as long as you know the nuances of the device driver you're trying to control.
But, I digress. The key point here is that I can open up a device, and I can make a DeviceIoControl() call, and true to form, I can use OVERLAPPED structures, and IO Completion Ports. That makes these calls "async", or with TINN, cooperative.
To wrap it up in a tidy little bow, here is a Device class which does the grunt work for me:
--[[ References http://msdn.microsoft.com/en-us/magazine/cc163415.aspx --]] local ffi = require("ffi") local bit = require("bit") local bor = bit.bor; local core_file = require("core_file_l1_2_0"); local core_io = require("core_io_l1_1_1"); local Application = require("Application") local IOOps = require("IOOps") local FsHandles = require("FsHandles"); local errorhandling = require("core_errorhandling_l1_1_1"); local WinBase = require("WinBase"); local Device = {} setmetatable(Device, { __call = function(self, ...) return self:open(...) end, }) local Device_mt = { __index = Device, } function Device.init(self, rawhandle) local obj = { Handle = FsHandles.FsHandle(rawhandle) } setmetatable(obj, Device_mt) Application:watchForIO(rawhandle, rawhandle) return obj; end function Device.open(self, devicename, dwDesiredAccess, dwShareMode) local lpFileName = string.format("\\\\.\\%s", devicename); dwDesiredAccess = dwDesiredAccess or bor(ffi.C.GENERIC_READ, ffi.C.GENERIC_WRITE); dwShareMode = bor(FILE_SHARE_READ, FILE_SHARE_WRITE); local lpSecurityAttributes = nil; local dwCreationDisposition = OPEN_EXISTING; local dwFlagsAndAttributes = FILE_FLAG_OVERLAPPED; local hTemplateFile = nil; local handle = core_file.CreateFileA( lpFileName, dwDesiredAccess, dwShareMode, lpSecurityAttributes, dwCreationDisposition, dwFlagsAndAttributes, hTemplateFile); if handle == INVALID_HANDLE_VALUE then return nil, errorhandling.GetLastError(); end return self:init(handle) end function Device.getNativeHandle(self) return self.Handle.Handle; end function Device.createOverlapped(self, buff, bufflen) local obj = ffi.new("FileOverlapped"); obj.file = self:getNativeHandle(); obj.OVL.Buffer = buff; obj.OVL.BufferLength = bufflen; return obj; end function Device.control(self, dwIoControlCode, lpInBuffer, nInBufferSize, lpOutBuffer, nOutBufferSize) local lpBytesReturned = nil; local lpOverlapped = self:createOverlapped(ffi.cast("void *", lpInBuffer), nInBufferSize); local status = core_io.DeviceIoControl(self:getNativeHandle(), dwIoControlCode, ffi.cast("void *", lpInBuffer), nInBufferSize, lpOutBuffer, nOutBufferSize, lpBytesReturned, ffi.cast("OVERLAPPED *",lpOverlapped)); local err = errorhandling.GetLastError(); -- Error conditions -- status == 1, err == WAIT_TIMEOUT (258) -- status == 0, err == ERROR_IO_PENDING (997) -- status == 0, err == something else if status == 0 then if err ~= ERROR_IO_PENDING then return false, err end end local key, bytes, ovl = Application:waitForIO(self, lpOverlapped); return bytes; end return Device
I've shown this kind of construct before with the NativeFile object. That object contains Read, and Write functions as well, but lacks the control() function. Of course the two could be combined for maximum benefit.
How to use this thing?
dev = Device("c:") dev:control(...)
OK, that's out of the way. Now, what about this change journal thing? Very simple now that the device is handled.
A change journal can look like this:
-- USNJournal.lua -- References -- http://msdn.microsoft.com/en-us/library/windows/desktop/aa364563(v=vs.85).aspx -- http://www.microsoft.com/msj/0999/journal/journal.aspx -- http://www.microsoft.com/msj/1099/journal2/journal2.aspx -- local ffi = require("ffi"); local bit = require("bit"); local bor = bit.bor; local band = bit.band; local core_io = require("core_io_l1_1_1"); local core_file = require("core_file_l1_2_0"); local WinIoCtl = require("WinIoCtl"); local WinBase = require("WinBase"); local errorhandling = require("core_errorhandling_l1_1_1"); local FsHandles = require("FsHandles"); local Device = require("Device") --[[ ChangeJournal An abstraction for NTFS Change journal management --]] local ChangeJournal = {} setmetatable(ChangeJournal, { __call = function(self, ...) return self:open(...); end, }); local ChangeJournal_mt = { __index = ChangeJournal; } ChangeJournal.init = function(self, device) local obj = { Device = device; } setmetatable(obj, ChangeJournal_mt); local jinfo, err = obj:getJournalInfo(); print("ChangeJournal.init, jinfo: ", jinfo, err) if jinfo then obj.JournalID = jinfo.UsnJournalID; obj.LowestUsn = jinfo.LowestValidUsn; obj.FirstUsn = jinfo.FirstUsn; obj.MaxSize = jinfo.MaximumSize; obj.MaxUsn = jinfo.MaxUsn; obj.AllocationSize = jinfo.AllocationDelta; end return obj; end ChangeJournal.open = function(self, driveLetter) local device, err = Device(driveLetter) if not device then print("ChangeJournal.open, ERROR: ", err) return nil, err end return self:init(device); end ChangeJournal.getNextUsn = function(self) local jinfo, err = self:getJournalInfo(); if not jinfo then return false, err; end return jinfo.NextUsn; end ChangeJournal.getJournalInfo = function(self) local dwIoControlCode = FSCTL_QUERY_USN_JOURNAL; local lpInBuffer = nil; local nInBufferSize = 0; local lpOutBuffer = ffi.new("USN_JOURNAL_DATA"); local nOutBufferSize = ffi.sizeof(lpOutBuffer); local success, err = self.Device:control(dwIoControlCode, lpInBuffer, nInBufferSize, lpOutBuffer, nOutBufferSize); if not success then return false, errorhandling.GetLastError(); end return lpOutBuffer; end function ChangeJournal.waitForNextEntry(self, usn, ReasonMask) usn = usn or self:getNextUsn(); local ReasonMask = ReasonMask or 0xFFFFFFFF; local ReturnOnlyOnClose = false; local Timeout = 0; local BytesToWaitFor = 1; local ReadData = ffi.new("READ_USN_JOURNAL_DATA", {usn, ReasonMask, ReturnOnlyOnClose, Timeout, BytesToWaitFor, self.JournalID}); local pusn = ffi.new("USN"); -- This function does not return until the USN -- record exits local BUF_LEN = ffi.C.USN_PAGE_SIZE; local Buffer = ffi.new("uint8_t[?]", BUF_LEN); local dwBytes = ffi.new("DWORD[1]"); local success, err = self.Device:control(FSCTL_READ_USN_JOURNAL, ReadData, ffi.sizeof(ReadData), Buffer, BUF_LEN); if not success then return false, err end local UsnRecord = ffi.cast("PUSN_RECORD", ffi.cast("PUCHAR",Buffer) + ffi.sizeof("USN")); return UsnRecord; end return ChangeJournal;
This very much looks like the change journal I created a few months back. The primary difference is the device control stuff is abstracted out into the Device object, so it does not need to be repeated here.
When we want to track the changes to the device, we make repeated calls to 'waitForNextEntry'.
local function test_waitForNextEntry(journal) local entry = journal:waitForNextEntry(); while entry do printJournalEntry(entry); entry = journal:waitForNextEntry(); end end
This is your typical serially written code. There's nothing that look special about it, no hint of async processing. Behind the covers though, way back in the Device:control() function, the actual sending of a command to the device happens using IO Completion Port, so if you're running with TINN, this particular task will 'waitForIO', and other tasks can continue.
So, using it in context looks like this:
local function main() local journal, err = ChangeJournal("c:") spawn(test_waitForNextEntry, journal); periodic(function() print("tick") end, 1000) end run(main)
In this case, I spawn the journal waiting/printing thing as its own task. Then I setup a periodic timer to simply print "tick" every second, to show there is some activity.
Since the journaling is cooperative (mostly waiting for io control to complete), the periodic timer, or UI processing, or what have you, is free to run, without any hindrance.
Combine this with the already cooperative UI stuff, and you can imagine how easy it could be to construct the scenario I set out to construct. Since all networking and file system operations in TINN are automatically async, it would be easy to log these values, or send them across a network to be stored for later analysis or what have you.
And there you have it. Async everywhere makes for some very nice scenarios. Being able to do async on any device, whether with standard read/write operations, or io control, makes for very flexible programming.
Next time around, I'm going to show how to do async DNS queries for fun and profit.
TINN Version 0.7 AvailablePosted: January 12, 2014 Filed under: Computicle, Core Programming, Lua, LuaJIT, System Programming, TINN | Tags: computicle, lua, luajit, tinn 4 Comments
Although TINN is under constant development, there's nothing like declaring a new "release". It's been 3 months since the 0.6 release. So, now there is a 0.7 release. You can read about the TINN v0.7 Release and install it if you would like.
There were 84 commits since the previous release, so I can't even remember all the changes that were involved. The major addition from my most recent work has to do with the new scheduler as described on this blog. That's the extensible, plug-in driven scheduler. Pretty nifty for my work at least.
There are quite a few additions, such as a revamped stream object, io completion port supported file interface, a logfile thing, and quite a few more interfaces from the OS.
Other items I have been working on include support for various COM interfaces such as DXGI, DXVA, MMDevice and some others. There are all in the "experimental" folder if you look at the enlistment. They are not quite ready for prime time, so they're not actually in the v0.7 release.
What can you do with TINN? In short, you can create all sorts of Windows based applications. Everything from scalable web services to interactive multi-tasking UI (including OpenGL based).
TINN is a command line tool (tinn.exe). As such, the easiest thing to do is bring up a command line shell and run various scripts through tinn.
c:\> tinn.exe hello.lua
The TINN repository contains numerous test cases that utilize the various modules of TINN.
That's it for now. Next release will be about cleanup and stabilization primarily.
Parallel ConversationsPosted: January 8, 2014 Filed under: Core Programming, Lua, LuaJIT, System Programming, TINN | Tags: event, lua, luajit, multi tasking, signal 1 Comment
I have been tinkering with words of late. I've added 'waitFor', 'when' and 'whenever' to the TINN lexicon, and my programming is becoming easier, more organized, and easier to describe.
Recently I added another set of words: 'waitSignal', 'signalOne', and 'signalAll'. If you were using another system, you might call these 'events', but really they're just signaling.
Here's how I might use them:
local Application = require("Application")(true) local function waiter(num) num = num or 0 local function closure() print(string.format("WAITED: %d",num)) end return closure; end local function main() for i=1,4 do onSignal(waiter(i), "waiting") end -- signal only one of them signalOne("waiting") -- sleep a bit giving other tasks -- a chance to run sleep(500) -- signal the rest of them signalAll("waiting")) sleep(2000) print("SLEEP AFTER signalAll") end run(main)
First, it spawns 4 tasks which will all be waiting on the 'waiting' signal. With 'signalOne', only 1 of the 4 waiting tasks will be awakened and continue execution. With the 'signalAll', the rest of the waiting tasks are all scheduled to run, and thus continue from where they left of.
The core primitive for signaling is the 'waitSignal()' function. This will essentially suspend the currently running task until the specified signal has been given. The 'onSignal()' function is a convenience which will spawn a separate task which will do the waiting and execute the specified function at the appropriate time.
If you were using another language, these might be 'onEvent()' and 'emit()'. Of course it's really easy to code up that way, just create a name alias to whatever you like. You could even do:
local function on(eventName, func) return onSignal(func, eventName) end
So, these are some more words that are in the parallel programming arsenal. In totality, the set is:
Time related
- sleep
- delay
- periodic
- waitFor
- when
- whenever
- onSignal
- signalOne
- signalAll
- waitSignal
Task and scheduler
- run
- stop
- spawn
- yield
With these words in hand, doing my multi-task programming is becoming easier by the moment. I don't worry about the lower level multi-tasking primitive such as mutexes, locks, and the like. I'm not really that concerned with timers, thread pools, or any of the other machinery that makes this all happen. I just write my simple code.
One thing has struck me though. This is all great for a single threaded environment. Basically collaborative multi-tasking. In this age of super fast CPUs, it turns out that going single threaded is just fine. In many cases it's actually better than going preemptive multi-threaded because you don't necessarily incur as much OS thread level context switching.
I'd like to go one step further though. A long time ago we didn't have L1/L2 caches on the CPUs, then they came along, got better, and now are fairly large. As a programmer, my concern for them is fairly minimal. I don't do much to ensure that the caches are used properly. I just write my code, and either the compiler, or the CPU itself deals with all the mechanics. Same goes with prefetching instructions and the like. I don't write my code to make that job any easier, it just happens automagically.
Here I am contemplating the same thing about threads. As my CPU has increasingly more cores, I am wondering if the CPU cycles should be viewed the same as we viewed levels of memory in the past. Is the a C1/C2 level of core execution. That is, most things should stay on the same thread because switching to a different core is costly. Something very low level should decide on when that switching occurs.
My application/runtime can know a lot about what I'm going to do because the thing is all laid out in script. After some analysis, perhaps I can decide "this thing is going to sleep for more than 500 ms, therefore I'm going to actually put it over here on this other thread which is actually sleeping.". In that way, I can reduce my resource usage because I won't be polling the timer every time through the scheduling loop, I can be smarter.
If I had such smartness, then when I have 16 – 100 cores, I'll be able to easily take advantage of those multiple cores without having to truly understand and navigate that madness on my own. Let the machine figure it out, it's much better at that than I am.
In the meanwhile, I've got these multi-tasking primitives which are making my life a lot easier when it comes to programming fairly complex systems.
Lua Coroutine RoundupPosted: January 3, 2014 Filed under: Core Programming, Lua, LuaJIT, System Programming, TINN | Tags: coroutine, lua, luajit, scheduler Leave a comment
My WordPress dashboard says this is my 250th post. I figure what better way to mark the occasion than to do a little bit of a roundup on one of my favorite topics.
One of the most awesome features of the Lua language is a built in co-routine mechanism. I've talked about co-routines quite a bit, doing a little series on the basics of coroutines, all the way up through a scheduler for multi-tasking.
Coroutines give the programmer the illusion of creating a parallel processing environment. Basically, write up your code "spawn" different coroutines, and then just forget about them… Well, almost. The one thing the Lua language does not give you is a way to manage those coroutines. Co-routines are a mechanism by which 'threads' can do cooperative multi-tasking. This is different from the OS level preemptive multitasking that you get with 'threads' on most OS libraries. So, I can create coroutines, resume, and yield. It's the 'yield' that gives coroutines the multi-tasking flavor. One co-routine yields, and another co-routine must be told to 'resume'. Something has to do the telling, and keep track of who's yielded, and who needs to be resumed. In steps the scheduler.
Summaries from my own archives
Lua Coroutines – Getting Started
Hurry Up and Wait – TINN Timing
Computicles – A tale of two schedulers
Parallel Computing is Child's Play
Multitasking single threaded UI – Gaming meets networking
Alertable Predicates? What's that?
Pinging Peers – Creating Network Meshes
From the Interwebs
GitHub Gist: Deco / coroutine_scheduler.lua
Beginner's Guide to Coroutines – Roblox Wiki
And of course the search engines will show you thousands more:
I think it would be really great if the Lua language itself had some form of rudimentary scheduler, but scheduling is such a domain specific thing, and the rudiments are so basic, that it's possibly not worth providing such support. The Go language supports a basic scheduler, and has coroutine support, to great effect I think.
The simple scheduler is simple, just do a round robbin execution of one task after the next as each task yields. I wrote a simple dispatcher which started with exactly this. Then interesting things start to happen. You'd like to have your tasks go into a wait state if they're doing an IO operation, and not be scheduled again until some IO has occurred. Or, you want to put a thread to sleep for some amount of time. Or, you want a thread to wait on a condition of one sort or another, or signals, or something else. Pretty soon, you basic scheduler is hundreds of lines of code, and things start to get rather complicated.
After going round and round in this problem space, I finally came up with a solution that I think has some legs. Rather than a complex scheduler, I promote a simple scheduler, and add functions to it through normal thread support. So, here is my final 'scheduler' for 2013:
local ffi = require("ffi"); local Collections = require("Collections"); local StopWatch = require("StopWatch"); --[[ The Scheduler supports a collaborative processing environment. As such, it manages multiple tasks which are represented by Lua coroutines. --]] local Scheduler = {} setmetatable(Scheduler, { __call = function(self, ...) return self:create(...) end, }) local Scheduler_mt = { __index = Scheduler, } function Scheduler.init(self, scheduler) local obj = { Clock = StopWatch(); TasksReadyToRun = Collections.Queue(); } setmetatable(obj, Scheduler_mt) return obj; end function Scheduler.create(self, ...) return self:init(...) end --[[ Instance Methods --]] function Scheduler.tasksArePending(self) return self.TasksReadyToRun:Len() > 0 end function Scheduler.tasksPending(self) return self.TasksReadyToRun:Len(); end function Scheduler.getClock(self) return self.Clock; end --[[ Task Handling --]] function Scheduler.scheduleTask(self, afiber, ...) if not afiber then return false, "no fiber specified" end afiber:setParams(...); self.TasksReadyToRun:Enqueue(afiber); afiber.state = "readytorun" return afiber; end function Scheduler.removeFiber(self, fiber) --print("DROPPING DEAD FIBER: ", fiber); return true; end function Scheduler.inMainFiber(self) return coroutine.running() == nil; end function Scheduler.getCurrentFiber(self) return self.CurrentFiber; end function Scheduler.step(self) -- Now check the regular fibers local task = self.TasksReadyToRun:Dequeue() -- If no fiber in ready queue, then just return if task == nil then return true end if task:getStatus() == "dead" then self:removeFiber(task) return true; end -- If the task we pulled off the active list is -- not dead, then perhaps it is suspended. If that's true -- then it needs to drop out of the active list. -- We assume that some other part of the system is responsible for -- keeping track of the task, and rescheduling it when appropriate. if task.state == "suspended" then return true; end -- If we have gotten this far, then the task truly is ready to -- run, and it should be set as the currentFiber, and its coroutine -- is resumed. self.CurrentFiber = task; local results = {task:resume()}; -- no task is currently executing self.CurrentFiber = nil; -- once we get results back from the resume, one -- of two things could have happened. -- 1) The routine exited normally -- 2) The routine yielded -- -- In both cases, we parse out the results of the resume -- into a success indicator and the rest of the values returned -- from the routine local success = results[1]; table.remove(results,1); --print("SUCCESS: ", success); if not success then print("RESUME ERROR") print(unpack(results)); end -- Again, check to see if the task is dead after -- the most recent resume. If it's dead, then don't -- bother putting it back into the readytorun queue -- just remove the task from the list of tasks if task:getStatus() == "dead" then --print("Scheduler, DEAD coroutine, removing") self:removeFiber(task) return true; end -- The only way the task will get back onto the readylist -- is if it's state is 'readytorun', otherwise, it will -- stay out of the readytorun list. if task.state == "readytorun" then self:scheduleTask(task, results); end end --[[ Primary Interfaces --]] function Scheduler.suspend(self, aTask) if not aTask then self.CurrentFiber.state = "suspended" return self:yield() end aTask.state = "suspended"; return true end function Scheduler.yield(self, ...) return coroutine.yield(...); end --[[ Running the scheduler itself --]] function Scheduler.start(self) if self.ContinueRunning then return false, "scheduler is already running" end self.ContinueRunning = true; while self.ContinueRunning do self:step(); end --print("FINISHED STEP ITERATION") end function Scheduler.stop(self) self.ContinueRunning = false; end return Scheduler
At 195 LOC, this is back down to the basic size of the original simple dispatcher that I started with. There are only a few functions to this interface:
- scheduleTask()
- step()
- suspend()
- yield()
- start()
These are the basics to do some scheduling of tasks. There is an assumption that there is an object that can be scheduled, such that 'scheduleTask' can attach some metadata, and the like. Other than that, there's not much here but the basic round robin scheduler.
So, how do you make a more complex scheduler, say one that deals with time? Well, first of all, I've also added an Application object, which actually contains a scheduler, and supports various other things related to an application, including add ons to the scheduler. Here's an excerpt of the Application object construction.
function Application.init(self, ...) local sched = Scheduler(); waitForIO.MessageQuanta = 0; local obj = { Clock = Stopwatch(); Scheduler = sched; TaskID = 0; wfc = waitForCondition(sched); wft = waitForTime(sched); wfio = waitForIO(sched); } setmetatable(obj, Application_mt) -- Create a task for each add-on obj:spawn(obj.wfc.start, obj.wfc) obj:spawn(obj.wft.start, obj.wft) obj:spawn(obj.wfio.start, obj.wfio) return obj; end
In the init(), the Application object creates instances for the add ons to the scheduler. These add-ons (waitForCondition, waitForTime, waitForIO) don't have to adhere to much of an interface, but you do need some function which is callable so that it can be spawned. The Application construction ends with the spawning of the three add-ons as normal threads in the scheduler. Yep, that's right, the add ons are just like any other thread, running in the scheduler. You might think; But these things need to run with a higher priority than regular threads! And yes, in some cases they do need that. But hay, that's an exercise for the scheduler. If the core scheduler gains a 'priority' mechanism, then these threads can be run with a higher priority, and everything is good.
And what do these add ons look like? I've gone over the before, but here's an example of the timing one:
local Functor = require("Functor") local tabutils = require("tabutils"); local waitForTime = {} setmetatable(waitForTime, { __call = function(self, ...) return self:create(...) end, }) local waitForTime_mt = { __index = waitForTime; } function waitForTime.init(self, scheduler) local obj = { Scheduler = scheduler; TasksWaitingForTime = {}; } setmetatable(obj, waitForTime_mt) --scheduler:addQuantaStep(Functor(obj.step,obj)); --scheduler:spawn(obj.step, obj) return obj; end function waitForTime.create(self, scheduler) if not scheduler then return nil, "no scheduler specified" end return self:init(scheduler) end function waitForTime.tasksArePending(self) return #self.TasksWaitingForTime > 0 end function waitForTime.tasksPending(self) return #self.TasksWaitingForTime; end local function compareTaskDueTime(task1, task2) if task1.DueTime < task2.DueTime then return true end return false; end function waitForTime.yieldUntilTime(self, atime) --print("waitForTime.yieldUntilTime: ", atime, self.Scheduler.Clock:Milliseconds()) --print("Current Fiber: ", self.CurrentFiber) local currentFiber = self.Scheduler:getCurrentFiber(); if currentFiber == nil then return false, "not currently in a running task" end currentFiber.DueTime = atime; tabutils.binsert(self.TasksWaitingForTime, currentFiber, compareTaskDueTime); return self.Scheduler:suspend() end function waitForTime.yield(self, millis) --print('waitForTime.yield, CLOCK: ', self.Scheduler.Clock) local nextTime = self.Scheduler.Clock:Milliseconds() + millis; return self:yieldUntilTime(nextTime); end function waitForTime.step(self) --print("waitForTime.step, CLOCK: ", self.Scheduler.Clock); local currentTime = self.Scheduler.Clock:Milliseconds(); -- traverse through the fibers that are waiting -- on time local nAwaiting = #self.TasksWaitingForTime; --print("Timer Events Waiting: ", nAwaiting) for i=1,nAwaiting do local fiber = self.TasksWaitingForTime[1]; if fiber.DueTime <= currentTime then --print("ACTIVATE: ", fiber.DueTime, currentTime); -- put it back into circulation -- preferably at the front of the list fiber.DueTime = 0; --print("waitForTime.step: ", self) self.Scheduler:scheduleTask(fiber); -- Remove the fiber from the list of fibers that are -- waiting on time table.remove(self.TasksWaitingForTime, 1); end end end function waitForTime.start(self) while true do self:step(); self.Scheduler:yield(); end end return waitForTime
The 'start()' function is what was scheduled with the scheduler, so that's the thread that's actually running. As you can see, it's just a normal cooperative task.
The real action comes from the 'yield()' and 'step()' functions. The 'yield()'s purpose in life is to suspend the current task (take it out of the active running list of the scheduler), and setup the appropriate stuff such that it can be activated later. The purpose of the "step()" function is to go through the list of tasks which are currently suspended, and determine which ones need to be scheduled again, because their time signal has been met.
And that's how you add items such as "sleep()" to the system. You don't change the core scheduler, but rather, you add a small module which knows how to deal with timed events, and that's that.
The application object comes along and provides some convenience methods that can easily be turned into globals, so that you can easily type things like:
delay(function() print("Hello, World!") end, 5000) -- print "Hello, World! after 5 seconds periodic(function() print("Hi There!") end, 1000) -- print "Hi, There!" every second
And there you have it. The core scheduler is a very simple thing. Just a ready list, and some algorithm that determines which one of the tasks in the list gets to run next. This can be as complex as necessary, or stay very simple.
Using an add on mechanism, the overall system can take on more robust scenarios, without increasing the complexity of the individual parts.
This simplified design satisfies all the scenario needs I currently have from high throughput web server to multi-tasking UI with network interaction. I will improve the scheduling algorithm, and probably make that a pluggable function for ease.
Lua coroutines are an awesome feature to have in a scripting language. With a little bit of work, a handy scheduler makes multi-task programming a mindless exercise, and that's a good thing.
Happy New Year!

subset: commoncrawl


text:
Olean, NY Hair and Nail Salon
Exclusively Yours
Exclusively Yours Hair & Nail Care is a full service salon in Olean, NY that can take care of all your beauty needs from head to toe—literally. We use only the best products from companies like Paul Mitchell, Enjoy, and Goldwell, and only the best stylists will work with you. So if you need a stylist for yourself or your whole wedding party, call our salon. If you need a gift for that special someone, we offer gift certificates for our salon and spa.
Exclusively Yours Offers These Services:
- Total hair care: highlights, lowlights, and coloring
- Color corrections
- Facials
- Body treatments
- Hand and foot treatments
- Waxing and hair removal
All facials and body wraps are performed by a N.Y.S. licensed esthetician.
We use 100% natural products and no MMAs ever.
Call Exclusively Yours today at 716-372-0003 to make an appointment. If you have any other questions or comments, please contact us at email@example.com.
Contact Information
208 West State Street
Olean, NY 14760-2538
Hours of Business:
Tuesday to Saturday
by appointment


text:
Earl Dean Finnicum
Earl Dean Finnicum, 87, went home to be with the Lord May 15, 2009, surrounded by his family after a short illness.
Born Aug. 9, 1921, in Carrollton, he was a son of the late John Franklin and Mary Ellen (Henry) Finnicum.
He married Kathryn Ratliff Dec. 2, 1950. He was a man who loved his family dearly. He was Christian by faith and attended the Southern Baptist Church.
He was a conductor on the railroad, retiring from Conrail in 1983. He was a WWII veteran, serving from Aug. 24, 1942, until Oct. 26, 1945, as a Medical Aidman. During his time in the Army, he received three bronze battle stars, a good conduct medal, meritorious service unit award and the European-African-Middle East Theatre Medal.
He is survived by his wife of 58 years, Kathryn; a son, Mark (Lisa and Tyler); daughter, Ellen; granddaughter, Keirsten; dog, Buddy and several nieces and nephews.
He was preceded in death by his parents; brothers and sisters-in-law, Paul and Marguerite, Eugene and Wilma and Carl and Peggy and half-sisters, Elizabeth Gotschall and Margaret Long.
The family wishes to express their gratitude to all the nurses, aides and staff of Schoenbrunn Healthcare Center for the loving care they gave Earl.
Funeral services were held May 18 in Allmon-Dugger-Cotton Funeral Home with Pastor Charlie Winter officiating. Burial followed in Westview Cemetery with full military honors by VFW, DAV and American Legion.
©2009 The Free Press Standard
Contact the webmaster at: email@example.com


text:
this wiki
Cattail is the upgrade for the Lily Pad which costs $10,000 to buy from Crazy Dave's Twiddydinkies and 225 sun to upgrade the Lily Pad in the Pool. It fires spikes at any lane at the firing speed of a Repeater. It becomes available for purchase the level before playing Level 4-5 in the first game.
Its tail and name are based on the Typha latifolia, the Cattail or reed, a perennial herbaceous plant in the genus Typha. Its face and head, however, are based on the domestic cat, Felis catus.
Suburban Almanac entry
Cattails can attack at any lane and shoot down balloon zombies too.
Must be planted on lily pads
"Woof!" says Cattail. "Woof woof woof! Does this confuse you? Do you expect me to say 'Meow' like a cat because the word 'cat' is in my name and I also look like a cat? That's not how things work around here. I refuse to be pigeonholed."
Cost: 225
Recharge: very slow
Plant Food upgrade (Plants vs. Zombies: All Stars) (China only)
Perceive weakness: Shoot 3 spikes to the enemy with lowest health. It requires Brave Plant Food to be activated.
Cattail has the damage and rate of fire of a Repeater. The spikes it fires home in on zombies and can damage zombies regardless of their position on the lawn. It attacks the zombies closest to the plant, although it prioritizes Balloon Zombies. However, they can only attack a few Shield Zombies around their Shields. It is very useful for killing Digger Zombies if they are in the back row, as it will kill them before they can even move.
Because a single Cattail can cover all six lanes, it makes a good early defensive plant. It also helps deal with Digger Zombies, Imps, and Balloon Zombies. However, they are easily overran because of the ability to cover all six lanes, as a Cattail may be targeting a Conehead Zombie, then a Pole Vaulting Zombie that is closer to the Cattail draws its fire, letting the Conehead Zombie go on unattacked. Also, all your Cattails will usually fire at the same target (except under certain conditions) and when that target dies, spikes in the air are wasted. This limits the effectiveness of large numbers of Cattails, so it is recommended that you don't rely solely on Cattails for pool defense.
It is a good idea to use Cattails in limited numbers in Survival Mode that take place in Pool or Fog. Do not forget to back them up with more consistent firing plants, such as Repeaters or Melon-pults.
Plants vs. Zombies Wiki has a gallery for Cattail.
Specific to Plants vs. Zombies
- Catttail's leaves's filenames in the game are "Cat_paw1.png", "Cat_paw2.png", and "Cat_paw3.png".
- Cattail is the only aquatic plant that is an upgrade and is also the only offensive upgrade plant that upgrades from an unoffensive plant. It is also the only aquatic plant that needs to be planted on Lily Pad.
- Cattail, Gloom-shroom, Split Pea, and Starfruit are the only plants who can shoot backwards. However, Cattail can attack any lane. Gloom-shroom can only attack one block near it, while Split Pea and Starfruit cannot attack other lanes backwards.
- Cattail does the damage of a pea and rate of fire like a Repeater, making it an aquatic version of a Repeater. However, its shots are homing. On the other hand, they cost 50 more sun (25 for the Lily Pad and 25 extra sun for the Cattail), and Cattails can get even more expensive in Survival: Endless.
- A Cattail can be downgraded to a Lily Pad if the player has a Pumpkin on it and try to dig up the Cattail, as that would cause the Pumpkin to fall into the water otherwise. Bungee Zombies can also downgrade a Cattail with a Pumpkin. This makes the Cattail the only plant able to downgrade.
- Cattail is the most expensive aquatic plant, costing 225 (25 extra for Lily Pad) sun with a 250 sun total. However, this value can rise in Survival: Endless due to the increasing price of upgrade plants.
- During the ending credits of Zombies on Your Lawn, it is shown next to Laura Shigihara's name.
- When a zombie is defeated, the spikes from Cattails are still aiming for the same direction, it will not damage the zombie behind the defeated one (unlike some other projectiles, which hit the zombies at the back).
- Cattail is a pun upon the Typha latifolia, commonly known as a cattail, as evidenced by Cattail's cat's head and its tail having a Cattail seed structure.
- Cattail's spikes moves slower than Cactus's.
- Cattail, Starfruit, Gloom-shroom, Cob Cannon, Cherry Bomb, Winter Melon or Melon-pult (splash damage), Doom-shroom, and Threepeater are the only plants that can hit zombies in other lanes.
- Cattail and Lily Pad are the only aquatic plants that can have a Pumpkin on them.
- Cattail is one of the two plants with a tail, the other one being the Cob Cannon.
- Cattail, Cactus, and Sea-shroom are the only projectile-shooting plants that use a projectile that is colored significantly differently from the plant itself.
- Cattail's spikes on the Nintendo DS version of the game will sometimes miss the zombies, and keep circling it until it dies, as well as not target zombies, but rather fly straight ahead and rotate slowly.
- Cattail has the longest range among the shooting plants.
- Cattail, Imitater, and Gatling Pea are the only plants with headwear.
- Cattail can fire in the most directions. The plant that fires in the second most directions is Gloom-shroom (eight) and the third is Starfruit (five).
- Cattail is one of four plants which can fire at a Digger Zombie, the others being Gloom-shroom, Split Pea, and Starfruit.
- Cattail is also one of three plants who can get rid of Balloon Zombies. The others are Blover and Cactus. However, Cattail is not listed as a weakness of Balloon Zombies in the Suburban Almanac entry.
- Cattail will not aim at a Snorkel Zombie. This is odd because it can fire anywhere on the screen and will aim at Balloon Zombies. This is possibly because it cannot see it. It may also be because it cannot fire at it as the Cattail might not be able to fire heights that low.
- Cattail, Marigold, and Starfruit are the only plants with one tooth.
- When a Cattail is targeting a Pole Vaulting Zombie and the zombie vaults, the spikes will act as it would if the zombie already died, but when the Pole Vaulting Zombie lands, the spike will turn around and attack the zombie again.
- This also happens when a Balloon Zombie loses its balloon or when a Dolphin Rider Zombie loses its dolphin.
- It is the fourth and last aquatic plant that can be unlocked.
- In the mini-game Heavy Weapon, Cattail's tail is pink instead of brown.
- It is strange how the Cattail can support a Pumpkin, as the Pumpkin looks like it is floating.
- The player cannot plant a Cattail on a Lily Pad with a plant besides a Pumpkin on it.
- If Cattail is planted in the very front rows, its spike cannot home into zombies.
- Cattail will not attack unless there are zombies in the second column or farther.
- Cattail only has a tongue as a plush toy.
- Cattail is the only upgrade plant that can be planted only in water.
- Cattail is the one of two plants in the game with a nose, the other being Pumpkin. However, Pumpkin's nose is just a hole, but Cattail's nose is real.
- In some versions, when a zombie targeted by a Cattail is not killed by it, the spikes fired targeting the dead zombie may circle around the dead zombie's death location. This behavior has been seen in the most recent Android update.
- It has a counterpart in Plants vs. Zombies 2, which is the Homing Thistle.
- Cattail seems to act like a normal plant on a Lily Pad, as they cannot plant anymore plants on the cattail and it can support a Pumpkin.
Specific to Plants vs. Zombies: All Stars
- Cattail's, Dogtail's, and Foxtail's elements are Earth, but they use the slingshot (element for water) as their weapon.
- Its spikes don't home in on zombies in this game. Also, it only shoots one spike at a time.
- It can only be planted on land in this game.
- It is not an upgrade in this game.
See also
|Plants vs. Zombies
|Plants vs. Zombies Adventures
|Plants vs. Zombies 2
|Player's House
|Ancient Egypt
|Pirate Seas
|Wild West
|Far Future
|Dark Ages
|Big Wave Beach
|Frostbite Caves
|Lost City
|Plants vs. Zombies 2 (Chinese version)
|Player's House
|Ancient Egypt
|Pirate Seas
|Wild West
|Kung-Fu World
|Far Future
|Dark Ages
|Big Wave Beach
|Frostbite Caves
|Monthly special
|Plants vs. Zombies Online
|Player's House
|Qin Shi Huang Mausoleum
|Ancient Egypt
|Pirate Seas
|Far Future
|East Sea Dragon Palace
|Winter Melon
|Endless Zone Power Ups
|Mini-game only
|Adventure Mode only
|Auxiliary plants
|Unused plants


text:
Which way is right? With Chris Ladd
Posts filtered on Tag
When Does Life Begin?
It is tough to make a credible argument that a viable fetus should be subject to abortion-at-will right up to the day of birth. It is equally difficult to justify full protection of the law, over the competing rights of a woman, to a collection of cells with a tail and gill slits. At any point in between there is no political position, whether through action or inaction, that doesn't potentially threaten the critical rights of either an unborn person or a woman. [Read More]


text: Washington Nationals pitcher Gio Gonzalez pitched six scoreless innings and added a solo home run in the Nats 3-0 win over the Miami Marlins Wednesday. A very impressive performance, no doubt, but it was Gonzalez's post-game analysis of his own pitching that really capped off a great night. MASN broadcaster Bob Carpenter asked Gonzalez if pitching in the low temperatures was difficult, and this was Gonzalez's answer:


text:
A refractory Metal Nitride and a refractory metal Silicon Nitride layer (64) can be formed using metal organic chemical deposition. More specifically, tantalum nitride (TaN) (64) can be formed by a Chemical Vapor Deposition (CVD) using Ethyltrikis (Diethylamido) Tantalum (ETDET) and ammonia (NH3). By the inclusion of silane (SiH4), tantalum silicon nitride (TaSiN) (64) layer can also be formed. Both of these layers can be formed at wafer temperatures lower than approximately 400° C. with relatively small amounts of carbon (C) within the film. Therefore, the embodiments of the present invention can be used to form tantalum nitride (TaN) or tantalum silicon nitride (TaSiN) (64) that is relatively conformal and has reasonably good diffusion barrier properties.
1. A method of forming a semiconductor device comprising the steps of:
forming a first insulation layer over a semiconductor device substrate; and
chemical vapor depositing a first refractory metal-semiconductor-nitride layer over the semiconductor device substrate after forming the first insulating layer; wherein a portion of the first insulating layer lies between a doped portion of the semiconductor device substrate and a portion of the first refractory metal-semiconductor nitride layer, and wherein the portion of the first insulating layer abuts the doped portion of the semiconductor device substrate and the portion of the first refractory metal-semiconductor nitride layer.
2. The method of
3. The method of
4. The method of
5. The method of
forming a second insulation layer prior to the formation of the first insulation layer, wherein the second insulation layer has an opening;
depositing a refractory metal layer after the step of forming a second insulation layer and prior to the formation of the first insulation layer;
depositing a second refractory metal-semiconductor-nitride layer using chemical vapor deposition after the step of depositing the refractory metal layer and prior to the formation of the first insulation layer; and
depositing a second conductive layer after the step of depositing the second refractory metal and prior to the formation of the first insulation layer.
6. The method of
7. The method of
8. The method of
9. The method of
10. The method of
the first insulating layer has an opening;
the method further comprises forming a first conductive layer after the first refractory metal-semiconductor-nitride layer; and
at least a portion of each of the first refractory metal-semiconductor-nitride layer and the first conductive layer lies within the opening.
11. The method of
12. The method of
13. The method of
14. The method of
15. The method of
16. The method of
placing the semiconductor device substrate into a chemical vapor deposition (CVD) reactor;
introducing a metal organic precursor into the CVD reactor;
introducing a semiconductor source into the CVD reactor; and
reacting the metal organic precursor, and the semiconductor source to form a refractory metal-semiconductor-nitride layer.
17. The method of
18. The method of
This application is a Division of Ser. No. 08/829,752, filed Mar. 31, 1997.
This is related to U.S. patent application Ser. No. 08/828,635, now U.S. Pat. No. 5,958,508; Ser. No. 08/828,638, now U.S. Pat. No. 5,888,588; Ser. Nos. 08/829,405; 08/831,286; and 08/831,287 all filed on Mar. 31, 1997, which are assigned to the current assignee hereof.
The present invention relates generally to the processing of semiconductor devices, and more specifically to providing a diffusion barrier onto a semiconductor device.
Modern semiconductor devices are requiring speeds in excess of 200 megahertz. In order to form future generations of semiconductor devices, copper (Cu) will essentially be required for interconnects. One problem with the use of copper is that copper cannot directly contact silicon dioxide because copper diffuses too easily through the silicon dioxide layer. Therefore, in the prior art the copper is typically surrounded by a diffusion barrier on all sides.
Diffusion barriers for copper include a number of materials, such as silicon nitride and various refractory metal nitrides (TiN, TaN, WN, MoN) and refractory silicon nitrides (TiSiN, TaSiN, WSiN), or refractory metal-semiconductor-nitride layers. Of all of these barriers, the two showing promise for barriers include tantalum nitride (TaN) and tantalum silicon nitride (TaSiN). These materials are usually deposited by sputtering. However, sputtering generally has poor sidewall step coverage, where step coverage is defined to be the percentage of a layer being deposited on a specific surface divided by the thickness of a layer being deposited on the uppermost surface of a semiconductor device. In the case of sputtered tantalum nitride (TaN) and tantalum silicon nitride (TaSiN), and the step coverage for a 0.35 μm via can be in the range of 5% to 20% for an aspect ratio of 3:1. Such low step coverage increases the risk that the barrier material will not be thick enough to be an effective diffusion barrier along the sides and bottom of a deep opening. In an attempt to get enough of the material along the walls of openings, a much thicker layer at the uppermost surface is deposited, however, this is undesirable because it increases the resistance of the interconnect.
Chemical vapor deposition (CVD) has been used to form tantalum nitride. The precursors for TaN includes tantalum halides, such as Tantalum Pentachloride (TaCl5). The problem with tantalum halides is that the halides react with the copper causing interconnect corrosion. Another precursor includes penta[dimethylamido]tantalum (Ta(NMe2)5). When this precursor is used to deposit tantalum nitride (TaN), the compound that is actually forms is an insulating layer of Ta3N5. An insulator cannot be used in contact openings or via openings because the insulator prevents electrical contact between the upper interconnect layer and the lower interconnect layer.
Still another known precursor includes terbutylimido-tris-diethyl amino tantalum [(TBTDET), Ta═NBu(NEt2)3]. This compound can be used to form TaN. However, there are problems associated with this precursor. Specifically, deposition temperatures higher than 600° C. is needed to deposit reasonably low resistivity films. Such high temperatures for back-end metallization are incompatible for low-k dielectrics and also induces high stresses due to thermal mismatch between the back-end materials. Another problem with the TBTDET precursor is that too much carbon (C) is incorporated within the layer. This compound generally has approximately 25 atomic percent carbon. The relatively high carbon content makes the layer highly resistive, and results in films that are less dense, lowering the diffusion barrier effectiveness for a comparable thickness of other materials. The resistivity of TaN when deposited using TBTDET at temperatures lower than 600° C. is approximately 12,000 μohm-cm. Films with such a high resistivity (desired is less than approximately 1000 μohm-cm) cannot be used for making effective interconnect structures.
CVD of titanium silicon nitride (TiSiN) has been demonstrated using titanium tetrachloride (TiCl4). This compound is again undesirable because in forming the TiSiN, chlorine is once again present which causes corrosion of copper and other materials used for interconnect.
A need, therefore, exists to deposit a TaN or TaSiN using organo-metallic precursors that can be formed relatively conformally with a reasonable resistivity and good barrier properties at lower wafer temperatures.
The present invention is illustrated by way of example and not limited in the accompanying figures, in which like references indicate similar elements, and in which:
FIG. 1 includes an illustration of a cross-sectional view of a portion of semiconductor device substrate after forming openings in an interlevel dielectric layer to doped regions within the substrate;
FIG. 2 includes an illustration of a cross-sectional view of FIG. 1 after forming materials needed to form interconnects in accordance with one embodiment of the present invention;
FIG. 3 includes an illustration of a substrate of FIG. 2 after forming inlaid interconnects to doped regions within the substrate;
FIG. 4 includes an illustration of a top view of the substrate of FIG. 3. after forming an interlevel dielectric layer and an opening within that layer;
FIG. 5 includes an illustration of a cross-sectional view of the substrate of FIG. 4 illustrating the opening to the lower interconnect;
FIG. 6 includes an illustration of a cross-sectional view of the substrate of FIG. 5 after forming an interconnect to a lower interconnect level; and
FIG. 7 includes in illustration of a cross-sectional view of the substrate of FIG. 6 after forming a substantially completed device.
Skilled artisans appreciate that elements in the figures are illustrated for simplicity and clarity, and have not necessarily been drawn to scale. For example, the dimensions of some of the elements in the figures are exaggerated relative to other elements to help to improve understanding of embodiment(s) of the present invention.
A refractory Metal Nitride and a refractory metal Silicon Nitride layer are formed using metal organic chemical deposition. More specifically, tantalum nitride (TaN) can be formed by a Chemical Vapor Deposition (CVD) using Ethyltrikis (Diethylamido) Tantalum [(ETDET), (Et2N)3Ta=NEt] and ammonia (NH3). By the inclusion of a semiconductor source such as silane (SiH4), a tantalum silicon nitride (TaSiN) layer can also be formed. Both of these layers can be formed at wafer temperatures lower than 500° Celsius with relatively small amounts of carbon (C) within the film. Therefore, the embodiments of the present invention can be used to form tantalum nitride (TaN) or tantalum silicon nitride (TaSiN) layer that is relatively conformal and has reasonably good diffusion barrier properties.
As used in this specification, chemical vapor deposition is one type of deposition method that is to be distinguished from sputter deposition. Sputter deposition is essentially a physical type of deposition in which a layer is being deposited onto a wafer by the action of a plasma directed toward a target. The material evolves from the target and is deposited in a substantially vertically orientation to the wafer. Chemical vapor deposition on the other hand is a chemical reaction that occurs at or near the surface of the substrate to form a layer along the exposed surfaces of the wafer.
Tantalum nitride (TaN) and tantalum silicon nitride (TaSiN) are formed using ETDET/NH3, and ETDET/NH3/SiH4 respectively. For TaN, the deposition generally takes place in a CVD reactor at a pressure in the range of 5-15 Torr. The monitored deposition temperature varies depending upon where the temperature is being monitored. If the heater block temperature is being monitored, the temperature is generally in a range of approximately 400-480° Celsius. If the wafer temperature is measured, the temperature is typically in a range of approximately 350-400° C.
The ETDET is introduced using Helium (He) as a carrier gas that is bubbled through the ampoule. The flow rate of the Helium (He) is in the range of 200-800 sccm. The heater box temperature for the ampoule is maintained at approximately 80° Celsius. In general, the heater box temperature can be maintained within a range of approximately 50-90° Celsius. The temperature of the ETDET within the ampoule is approximately about 10° Celsius lower than the heater box temperature. Ammonia (NH3) is introduced at a rate ranging from 200-500 sccm which generally gives a deposition rate of approximately 150-200 Å/minute. The deposition rate will also depend on the reactor configuration. Using these parameters TaN film can be deposited that has less than 15% carbon (C) and is generally no more than 1%. When used as a barrier layer, the layer TaN is usually deposited to a thickness in a range of approximately 200-300 Å along an exposed surface of the substrate, and generally has step coverage of greater than 50 % at the bottom surface of an opening having an aspect ratios of 3:1.
The flow of ammonia has been observed to enhance the deposition across all temperature ranges. Without ammonia limited or no deposition is observed even at high wafer temperatures. This is in contrast to the precursor (TBTDET) used to deposit TaN as reported in literature, which reported deposition without ammonia (NH3).
In CVD systems, there is typically more difficulty depositing the layer at the bottom of the opening, and therefore, the step coverage at the bottom is a good indicator of the thinnest portion of the film. The TaN also has been found to have reasonably good adhesion to the surfaces of both metals and oxides. This is important for integrating the layer into an interconnect process. Should the layer be used to make contact (electrical or physical) to a Silicon containing layer such as a gate electrode or doped regions within a semiconductor substrate, titanium may be deposited between the TaN and Silicon (Si) to form a good ohmic contact. Without the titanium, a relatively high contact resistance between TaN and p+ Silicon may be formed because of large differences in the work functions between p+ silicon and tantalum nitride.
The deposition parameters for TaSiN are the same except as noted below. The pressure is typically in a range of approximately 0.1 to 1 Torr. The flow rates are slightly changed in that: the Helium (He) flows at a rate of approximately 50-150 sccm at the same conditions for the ampoule as described earlier for TaN; ammonia (NH3) is introduced at a rate of approximately 150-300 sccm; and silane (SiH4) is introduced at approximately 1-10 sccm. These parameters give a deposition rate of approximately 150-250 Å/minute with approximately the same carbon incorporation and adhesion characteristics as the TaN.
Different source gases may be used for the Silicon source and the TaN precursor. Specifically, it is possible that Disilane (Si2H6) or some other Silicon gas could be used. In addition, it is believed that source gasses containing other semiconducotr souces, such as germanium will work as well. However, care should be exercised, in assuring that gas phase reaction is not present. Also, the wafer temperature of the deposition should not exceed 500° Celsius, and be typically less than 400° Celsius, because of the issues noted previously. The TaN precursor has similar concerns. In general, the ethyl group attached to the doubly bonded nitrogen can comprise either an ethyl [(Et2N)3Ta═NEt] or a methyl [(Et2N)3Ta═NMe] group. The carrier gas for the ampoule includes helium (He), argon (Ar), nitrogen (N2) or hydrogen (H2).
Following deposition of a TaN films by CVD, the films can be exposed to an in-situ plasma treatment which allows a reduction in resistivity of the deposited films. Different gases, including argon, hydrogen, nitrogen, silane, and ammonia, individually or in combination can be used for plasma treatment. For example, the use of argon generally allows a reduction in resistivity of the films by a factor of 2 or more. Other gases generally work best in combination with argon although this is not necessary. Use of silane will allow incorporation of Si in the films, thus forming TaSiN in the matrix. This method allows control over the Si to N ratio in the film. The flow rate for the gases can range from 100-1000 sccm; pressure ranging from 100 mTorr-15 Torr; and plasma power ranging from 100-2000 Watts. The plasma treatment can also be done intermittently, that is, deposition/plasma/deposition steps. Further, thermal annealing of films by SiH4 can also be done instead of using plasma in order to incorporate Si in the films. The process involves flowing SiH4 flow over the heated wafers after deposition step. Anneal conditions similar to plasma can be used except that there would be no plasma.
Embodiments of the present invention are better understood with the example that follows in which two levels of interconnects are formed using the chemically vapor deposited material. FIG. 1 includes an illustration of a cross-sectional view of a portion of a semiconductor device substrate 10 before interconnects are formed. The semiconductor device substrate 10 is a monocrystalline semiconductor wafer, a semiconductor-on-insulating wafer, or any other substrate used to form semiconductor devices. Field isolation regions 12 are formed over the semiconductor device substrate 10. Doped regions 14 are source/drain regions for a transistor and lie within the substrate 10 adjacent to the field isolation regions 12. A gate dielectric layer 22 and gate electrode 24 overlie the substrate 10 and portions of the doped regions 14. An interlevel dielectric layer 26 is deposited over the semiconductor device substrate 10. The interlevel dielectric layer 26 can include an undoped, a doped, or combination of doped and undoped silicon dioxide films. In one particular embodiment, an undoped silicon dioxide film is covered by a borophosphosilicate glass (BPSG) layer. After planarization of layer 26, openings 28 are formed through the interlevel dielectric layer 26 and extend to the doped regions 14. As illustrated in FIG. 1, the openings 28 include a contact portion which is relatively narrow that contacts the doped regions 14, and a relatively wider interconnect trench, which is where the interconnect is formed. In one example of FIG. 1, the contact portion has an aspect ratio is 3:1 as compared to the trench. This is an example of a dual damascene process for forming inlaid interconnects which are generally known within the prior art.
The materials used to form the contacts and interconnects are then deposited over the interlevel dielectric layer 26 and within the openings 28. As illustrated in FIG. 3, which illustrates a partially completed device, layer 32 of titanium or other refractory material is formed, and is in contact with the doped regions 14. This layer generally has a thickness in a range of approximately 100-400 Å. Next, a TaN or TaSiN layer 34 is formed over layer 32. The tantalum nitride layer 34 or TaSiN layer 34 is formed using the previously described deposition parameters. The thickness of the layer is in the range of approximately 200 to 300 Å. A conductive layer 36 is formed within the remaining portions of the openings and overlying 34. The conductive layer 36 typically includes copper (Cu), aluminum (Al), tungsten (W) or the like. In this particular embodiment, the conductive layer 36 is copper. The partially completed device is then polished to remove the portions of layers 32, 34 and 36 that overlie the interlevel dielectric layer 26. This forms contact portions and interconnect portions for the interconnects 44 and 42 as illustrated in FIG. 3.
A second interlevel dielectric layer 56 is deposited and patterned over the interconnects 42 and 44 and the first interlevel dielectric layer 26. FIGS. 4 and 5 illustrate top and cross-sectional views, respectively, of the second interlevel dielectric layer after patterning. The second interlevel dielectric layer 56 includes a doped or undoped oxide. The patterning forms a via opening 52 and an interconnect trench 54. Other via openings and interconnect trenches are formed but are not shown in FIGS. 4 and 5.
As illustrated in FIG. 6, TaN or TaSiN layer 64 is then deposited using one of the previously described deposition techniques. Layer 64 contacts the lower interconnect 42. Layer 64 has a thickness in a range of approximately 200 to 300 Å and is covered by a second conductive layer 66 using a material similar to layer 36. The portions of the layers 64 and 66 overlying the second interlevel dielectric layer outside of the interconnect trench are then removed by polishing to give the structure as illustrated in FIG. 6. The combination of layers 64 and 66 forms a bit line 62 for the semiconductor device. A substantially completed device 70 is formed after depositing a passivation layer 72 overlying the second level interconnects, as illustrated in FIG. 7. In other embodiments, other insulating layers and interconnects levels can be formed but are not shown in the figures.
Many benefits exist for embodiments of the present invention. The CVD reaction that forms TaN or TaSiN is performed at wafer temperatures lower than approximately 500° Celsius, and typically less than 400° Celsius. Therefore, the process is compatible with low-k dielectrics and does not induce high stresses in the films. The amount of carbon incorporation is less than 15 atomic percent, and typically 1 atomic % or less. Therefore, the film does not have porous qualities and is a better diffusion barrier compared to using TBTDET as a precursor. The reduced carbon results in a resistivity of the CVD TaN films that are at least an order of magnitude lower compared to the prior art use of the TBTDET. Still, other advantages with the embodiments is the relative ease of integration into an existing process flow.
Thus it is apparent that there has been provided, in accordance with the invention, a process for depositing a diffusion barrier to fabricate a semiconductor device, which fully meets the advantages set forth above. Although the invention has been described and illustrated with reference to specific illustrative embodiments thereof, it is not intended that the invention be limited to those illustrative embodiments. Those skilled in the art will recognize that variations and modifications can be made without departing from the spirit of the invention. It is therefore intended to include within the invention all such variations and modifications as fall within the scope of the appended claims and equivalents thereof.
Citations de brevets
Citations hors brevets
Référencé par
Événements juridiques
Image d'origine
Faire pivoter


text:
Antonio Conte's men wrapped up the Scudetto against the Sicilian club in a match where Pogba was sent off after he reacted badly to a push by Salvatore Aronica.
The former Manchester United man spat at the Palermo defender to earn a straight red card in the 83rd minute.
As a result of Monday's league ruling, the 20-year-old France international will miss Juventus' final three matches of the campaign against Atalanta, Cagliari and Sampdoria.


text:
Languages and dialects spoken in Cameroon
Found 214 language names
Showing items 201 to 214 of 214
This list shows the primary names of the languages and dialects.
This list shows only languages and dialects with audio or video materials available.
This list shows only languages indigenous to this location.
Tuki [Centre]
Tum [Centre]
Tunen [Centre]
Vute [Centre]
Wando [Nord-Ouest]
Weh [Nord-Ouest]
Wula [Cameroon]
Wushi [Nord-Ouest]
Xadi [Nord-Ouest]
Yamba: Mbem [Nord-Ouest]
Yamba: Mfe [Nord-Ouest]
Yamba: Nkot [Nord-Ouest]
Yive [Sud-Ouest]
Zumaya [Extrème-Nord]
Show pages: «Previous 1 2 3 [Show All]
News about Cameroon
Training recordists in Africa: This recordist training course in west Central Africa gives some great insight into the recording process used by Global Recordings Network.
Audiovie Cameroun: General information, articles, news and contact details about GRN Cameroon.


text:
Beyonc?'s collaborators may be interested in taking shots at Kelly Rowland on her behalf, but Beyonc? herself has no interest in female-star infighting. In the cover story for the August/September 2011 issue of Complex, she refused to provide any grist about Rowland or any other female pop stars for the ever-churning rumor mill.
Interviewer Gabriel Alvarez tries to hedge a bit about rumors. "The fact that both released singles on the same day back in ?08 was somehow interpreted to mean that Bey was trying to sabotage Kelly," he notes, implying a conflict on the level of Blur and Oasis in 1995, even while denying its veracity. In fact, Rowland's "single" was actually the digital-only deluxe reissue of Miss Kelly, and Beyonc?'s "release" was "Sweet Dreams," leaked the day after it was recorded, and months before the release of I Am…Sasha Fierce, so even a Beyonc? collaborator who might identify with Bhasker's Twitter outburst yesterday wouldn't have purposefully made the song public.
To be fair, when Alvarez asks about drama, he does observes that "you can tell the question irks her, simply in the asking." Beyonc?'s response, though, shuts the line of questioning down, despite Alvarez's suggestion of "not-at-all subtle lines of distinction":
There is room on this earth for many queens. I have an authentic, God-given talent, drive, and longevity that will always separate me from everyone else. I?ve been fortunate to accomplish things that the younger generation of queens dream of accomplishing. I have no desire for anyone else?s throne. I am very comfortable in the throne I?ve been building for the past 15 years.


text:
H2O Express, Ltd., a Texas limited partnership, was established in 2002 with headquarters in Fort Worth, Texas.
H2O Express strives to provide safe, good tasting, purified drinking water to Texas households at affordable prices.
Currently, H2O Express has locations in the Dallas-Fort Worth metropolitan area as well as Central and West Texas.