[BUG] Coredns cannot resolve node hostname #1087
Comments
ccjjxx99
changed the title
|
@ccjjxx99 Thank you for posting issue. coredns can not resolve node name, i am afraid that maybe you have not used the correct coredns instance, so please check the following points:
|
|
@rambohe-ch Thanks for your response. I checked these three points:
then I check the log of coredns: I can see the first item shows NOERROR to resolve |
@ccjjxx99 please check the contents of /etc/resolv.conf file in metrics-server pod mount namespace. the step as following:
by the way, as the output of |
|
by the way, the logs of coredns also showed that coredns can resolve |
|
@rambohe-ch I tried various methods but all failed to get into the metrics - server container. But I think this is not the problem. I think now the question is why coredns can resolve |
@ccjjxx99 because there are no
you can check the contents of |
|
@rambohe-ch Thank you for your patience to answer. Now I can see the resolv.conf: 10.96.0.10 is my kube-dns service IP. So we can't see the request is forwarded to which pod form the Refer to the article https://cloud.tencent.com/developer/article/1669860 .Now I think the problem is that the dns resolve request form metric-server sent to coredns is |
@ccjjxx99 I think that the contents of |
|
@rambohe-ch I checked service topology documentation. My understanding is that the following three operations are required to implement the coredns service topology:
I think I have completed the steps needed to turn on the service topology. The coredns log on the edge node is shown below. From here we can see that There are no requests for any network segment that is 10.244.0.xxx. Coredns on the edge node only receives the dns resolve requests from other edge node's pods. |
|
@ccjjxx99 you should check the network setting of by the way, you can deploy a test pod(like busybox) on cloud node, so you nsenter into test container, then use the following commands to check dns requests fowrading:
|
|
@rambohe-ch Thanks a lot. Now I believe that you are right. I started a So kube-dns server forward my dns resolve request to other coredns pods rather than the coredns on cloud node ( ip 10.244.0.25).
Are there any other steps needed? Does the daemonset configuration of coredns need to be adjusted? Thanks a lot! |
@ccjjxx99 please check the networking setting of maybe you can try to restart kube-proxy pod on the cloud node, and check service topology capability can works or not. by the way: A openyurt community meeting on 11:00AM(beijing time) 2022.12.7 will be held, and we will introduce the details of data filter framework of yurthub(include service topology capability), and welcome you to the meeting.
|
|
@rambohe-ch I checked my kube-proxy work mode in configmap, it's empty. Refering to https://kubernetes.io/docs/reference/config-api/kube-proxy-config.v1alpha1/#kubeproxy-config-k8s-io-v1alpha1-ProxyMode , I know the default mode is So I checked the iptables rules. It seems to contain the coredns pod ip of all edge nodes (like When I change the kube-proxy mode to So the service topology doesn't work. How can I fix it? Thank you very much. |
@ccjjxx99 Please check whether yurthub component with cloud working mode is also deployed on the node where Metric-server is deployed? because the service topology capability is provided by yurthub component. |
|
@rambohe-ch My cloud node's name is yurt-app-manager, yurt-controller-manager, yurt-tunnel-dns, yurt-tunnel-server are all work without error. |
@ccjjxx99 The reason is already obvious, Yurthub component is not deployed on the cloud node. and i suggest that divide kubernetes control-plane components(like etcd, kas, kcm, etc.) and OpenYurt control-plane components(like ycm, yurt-tunnel etc.) into two nodes. and deploy Yurthub with cloud working mode on cloud node together with OpenYurt control-plane components. |
|
@rambohe-ch 原来如此
I have only one cloud server. Can I deploy yurthub on the cloud node with kubernetes control-plane components? The command bellow to start yurthub in the container seems to be to join the cluster. However, my cloud node itself is the k8s control-plane node of the cluster. And thank you very much for your patient answers. |
@ccjjxx99 yes, yurthub can be deployed on the kubernetes control-plane node, and you need to execute the following steps:
|
|
@rambohe-ch Thanks a lot! It works. I successfully deployed yurthub on the master node and the service topology function works well. My metrics-server is also working fine. Thanks again for your answer! |
|
@ccjjxx99 I will close this issue because the problem had been solved, and you can feel free to reopen this issue if you need. |
|
@ccjjxx99 would you mind registering as an OpenYurt user in this issue? and your registration will encourage us to improve OpenYurt better. |
|
If I want to use edge hostname to visit service, Do I need to mounted the yurt-tunnel-nodes configmap to coredns? |
@TonyZZhang yes, you need to mount yurt-tunnel-nodes configmap into coredns container. but we have recommended to install a independent coredns instance(named yurt-tunnel-dns: https://github.com/openyurtio/openyurt/blob/master/config/setup/yurt-tunnel-dns.yaml) for hostname resolution on cloud nodes or master nodes. |
What happened:
I have deployed metrics-server on the cloud node. It continues to report the following error:
When I turned on the log function of coredns and checked the logs, I found that coredns could not resolve the hostname:
In fact, I have mounted the yurt-tunnel-nodes configmap to coredns:
And I added the
hoststo the configmap of coredns:And the yurt-tunnel-nodes configmap is as shown below, where 10.107.2.246 is the ClusterIP of x-tunnel-server-internal-svc:
I think all these configures is well. So why coredns returns NXDOMAIN where solving the node hostname?
What you expected to happen:
Coredns can resolve the node hostname to the ClusterIP of x-tunnel-server-internal-svc.
How to reproduce it (as minimally and precisely as possible):
Anything else we need to know?:
Environment:
kubectl version): 1.22.8cat /etc/os-release): Ubuntu 22.04.1 LTSuname -a): 5.15.0-46-genericothers
/kind bug
The text was updated successfully, but these errors were encountered: