ObReferenceObject(fileObject) causing REFERENCE_BY_POINTER (18) bugcheck

We have a file system minifilter driver where we have registered dispatch function for IRP_MJ_PNP, IRP_MJ_DEVICE_CONTROL, IRP_MJ_POWER etc. My driver is crashing randomly while incrementing the reference count using ObReferenceObject. In IRP_MJ_DEVICE_CONTROL dispatch function, i am getting crash. Below is the crash dump-
REFERENCE_BY_POINTER (18)
Arguments:
Arg1: 0000000000000000, Object type of the object whose reference count is being lowered
Arg2: ffffd10e133d8d50, Object whose reference count is being lowered
Arg3: 0000000000000010, Reserved
Arg4: 0000000000000001, Reserved
The reference count of an object is illegal for the current state of the object.
Each time a driver uses a pointer to an object the driver calls a kernel routine
to increment the reference count of the object. When the driver is done with the
pointer the driver calls another kernel routine to decrement the reference count.
Drivers must match calls to the increment and decrement routines. This BugCheck
can occur because an object's reference count goes to zero while there are still
open handles to the object, in which case the fourth parameter indicates the number
of opened handles. It may also occur when the object's reference count drops below zero
whether or not there are open handles to the object, and in that case the fourth parameter
contains the actual value of the pointer references count.

STACK_TEXT:
fffffb0a653affe8 fffff80717685c35 : 0000000000000018 0000000000000000 ffffd10e133d8d50 0000000000000010 : nt!KeBugCheckEx
fffffb0a653afff0 fffff807108d76e2 : ffffd10e0b9c7010 0000000000000001 0000000020206f49 ffffd10e0254d2c0 : nt!ObfReferenceObject+0x182d55
fffffb0a653b0030 fffff80717544799 : ffffd10e0b9c7010 ffffd10e0254d2c0 0000000000000001 0000000000000000 : myDriver!function+0x4a
fffffb0a653b00a0 fffff80717aa52a1 : fffffb0a653b0440 ffffd10e0b9c7010 0000000000000001 ffffd10e0254d2c0 : nt!IofCallDriver+0x59
fffffb0a653b00e0 fffff80717a88ec1 : ffffd10e00000005 ffffd10e0254d2c0 fffffb0a20206f49 fffffb0a653b0440 : nt!IopSynchronousServiceTail+0x1b1
fffffb0a653b0190 fffff80717af1cf6 : 000000003861e3e8 0000000000000ca0 0000000000000000 000000003861e480 : nt!IopXxxControlFile+0xe61
fffffb0a653b02e0 fffff8071767bcc5 : ffffd10dd6f61080 000000003861e3c8 fffffb0a653b0368 0000000000000001 : nt!NtDeviceIoControlFile+0x56
fffffb0a653b0350 00007ffe9ebf00b4 : 00007ffe9b7aa6a6 000000000f360fa0 0000000000880320 0000069d9b205e7c : nt!KiSystemServiceCopyEnd+0x25

Below is the device object structure while crash happened-
[+0x000] Type : 3 [Type: short]
[+0x002] Size : 0x218 [Type: unsigned short]
[+0x004] ReferenceCount : 0 [Type: long]
[+0x008] DriverObject : 0xffffd10dd5a90e30 : Driver "\FileSystem\myDriver" [Type: _DRIVER_OBJECT *]
[+0x010] NextDevice : 0xffffd10ddf04c4b0 [Type: _DEVICE_OBJECT *]
[+0x018] AttachedDevice : 0x0 [Type: _DEVICE_OBJECT *]
[+0x020] CurrentIrp : 0x0 [Type: _IRP *]
[+0x028] Timer : 0x0 [Type: _IO_TIMER *]
[+0x030] Flags : 0x80 [Type: unsigned long]
[+0x034] Characteristics : 0x0 [Type: unsigned long]
[+0x038] Vpb : 0x0 [Type: _VPB *]
[+0x040] DeviceExtension : 0xffffd10e133d8ea0 [Type: void *]
[+0x048] DeviceType : 0x2 [Type: unsigned long]
[+0x04c] StackSize : 30 [Type: char]
[+0x050] Queue [Type: ]
[+0x098] AlignmentRequirement : 0x0 [Type: unsigned long]
[+0x0a0] DeviceQueue [Type: _KDEVICE_QUEUE]
[+0x0c8] Dpc [Type: _KDPC]
[+0x108] ActiveThreadCount : 0x0 [Type: unsigned long]
[+0x110] SecurityDescriptor : 0xffff9703e1979920 [Type: void *]
[+0x118] DeviceLock [Type: _KEVENT]
[+0x130] SectorSize : 0x0 [Type: unsigned short]
[+0x132] Spare1 : 0x0 [Type: unsigned short]
[+0x138] DeviceObjectExtension : 0xffffd10e133d8f68 [Type: _DEVOBJ_EXTENSION *]
[+0x140] Reserved : 0x0 [Type: void *]

In device object, I can see that flags value is 0x80 which means DO_DEVICE_INITIALIZE. This flag set when we create device object using IoCreateDevice but flag should clear once initialization is completed. But flag is still set and we are incrementing the reference count. Can it be the reason of the crash?
Any help in this topic would be appreciated. Thanks in advance.