Can a device be tracked if bluetooth is enabled but visibility disabled? Is it possible to track a Bluetooth signal of Bluetooth enabled Android phone with disabled visiblity? Here, tracking refers to detecting a nearby device. As an application, if an attacker has many tracking devices all across the city (e.g. public transportation), they could always know when you pass these points.
I have heard of professional tracking devices, which are able to track/locate Bluetooth devices, even if their visibility is turned off.
By design, it's not supposed to be possible to actively get the address of a Bluetooth device (which is then used to identify the device for tracking purposes) when it's not in discoverable mode, as it won't respond to inquiries. However, if a device is actively communicating with another device, it will be transmitting information that can be used to track it passively. Exactly what information is available depends on which BT features the device is using. Later BT versions include security features that can make address detection, and thus tracking, harder, though these features are generally optional, so newer devices may still be as easy to track as older devices.

In older BT versions, part of the address, the LAP (Lower Address Part; assigned by the manufacturer and unique to the device) is broadcast in cleartext in each packet. Frequency hopping can make it hard to capture packets, but specialized receivers can counter this. Newer BT versions can generate (and periodically change) a random address instead of using the device's; this behavior is called LE Privacy1. 

Another part of the address, the UAP (Upper Address Part; part of the manufacturer's ID assigned to the manufacturer), is used for various algorithms, such as error correction and inter-packet timing, and can be calculated2.

A device also sends its full address in cleartext when authenticating with another device. An attacker can try to capture this by disconnecting the devices (using spoofed packets with a specialized payload) and then listening for the reconnect.

Note all of these attacks require that a device is connected to some other device. If a device is undiscoverable and unpaired, it won't be transmitting and can't be made to transmit. Even then, some types of tracking are possible.

An app on your phone can use its greater access to leverage Bluetooth to track your location, not based on info from the phone's own data but on nearby BT devices. Google's Android location services3 gathers info on nearby BT beacons (even if not paired) and sends it to Google. Note that this is a different type of tracking than the other techniques allow.

Reference:


Understanding Bluetooth Security; M. Loveless; 2018-01-09
Discovering the Bluetooth UAP; M. Ossmann; 2014-06-10
Google can still use Bluetooth to track your Android phone when Bluetooth is turned off; D. Yanofsky; 2018-01-24
Hacking Bluetooth; E. Chai, B. Deardorff, C Wu; 2012-12-09