This Site Is My Tech Obstacle Log, Leave Me A Msg If U ...

I have read through [“FATAL: No nodes returned from search!” while searching or accessing node using SSH]

But it doesn’t help

I have similar case: 1. knife node list can show current running nodes 2. knife search has no result

but not the same that: 1. i can add new nodes and run chef-client on clients 2. i can run chef-client cmd on exisiting client nodes

Since when: My chef has been non-stop generating on system email to /var/spool/mail/root and so the disk was 100%. As everyone knows when you have disk full/CPU 100%/Memory 100% you can have strange behavior. My result was my chef cannot search client node since then.

knife search node name:* FATAL: No nodes returned from search!

but my knife node list has expected result, and knife node show <node name>:

[root@ip-10-0-0-10 chef-repo]# knife node list i-4f634feb i-dc286552 [root@ip-10-0-0-10 chef-repo]# knife node show i-4f634feb Node Name: i-4f634feb Environment: _default FQDN: ip-10-0-4-94.ap-southeast-1.compute.internal IP: 10.0.4.94 Run List: role[chefclient-nginx-gl-microcache] Roles: Recipes: drupalsetting Platform: amazon 2015.03 Tags: [root@ip-10-0-0-10 chef-repo]# knife node show i-dc286552 Node Name: i-dc286552 Environment: _default FQDN: ip-10-0-0-214.ap-southeast-1.compute.internal IP: 10.0.0.214 Run List: role[chefclient-nginx-gl-microcache] Roles: chefclient-nginx-gl-microcache Recipes: git, build-essential, basicservice, php56, php56-fpm, nginx, microcache, composer, deploycode, diskmount, drupalsetting, newrelic, newrelic::php_agent Platform: amazon 2015.03 So we can can the search function of knife and node list function are separated. But how can i resync those running information to server? the trick is running chef-client cmd to play cookbook on client node:

chef-client -o 'recipe[drupalsetting]'

After running the above command you can see the info is re-sync. Although the Roles: is empty, you can still call knife ssh 'role:ROLE_NAME' "CMD" to run command on all client nodes.

You can see those nodes are back.

[root@ip-10-0-0-10 chef-repo]# knife search node name:* 2 items found Node Name: i-4f634feb Environment: _default FQDN: ip-10-0-4-94.ap-southeast-1.compute.internal IP: 10.0.4.94 Run List: role[chefclient-nginx-gl-microcache] Roles: Recipes: drupalsetting Platform: amazon 2015.03 Tags: Node Name: i-dc286552 Environment: _default FQDN: ip-10-0-0-214.ap-southeast-1.compute.internal IP: 10.0.0.214 Run List: role[chefclient-nginx-gl-microcache] Roles: Recipes: drupalsetting Platform: amazon 2015.03 Tags: One thing i leart from this issue: When removing nodes, rememeber to run both “knife node delete <node name>” and “knife client delete <node name>” at the same time! otherwise you may have too many deprecated client nodes that create unexpected problem!